Build an Offline Voice-Controlled Smart Home Hub with ESP32-S3 & INMP441

2. Overview & Engineering Objectives

Commercial smart speakers (Amazon Alexa, Google Home) require constant high-speed internet connections, introduce latency, and raise privacy concerns by streaming raw domestic audio to remote cloud servers. This project builds a standalone, 100% offline, zero-subscription voice automation hub powered by the ESP32-S3’s native vector acceleration instructions.

Core Engineering Capabilities:

Digital I2S Audio Pipeline: Captures uncompressed 16-bit 16kHz PCM audio directly from an INMP441 digital MEMS microphone, completely bypassing noisy analog ADC converters.

Privacy-First Offline Execution: All acoustic keyword analysis and command decoding occur locally in on-chip SRAM/PSRAM without transmitting audio packets over the internet.

Multi-Channel Appliance Actuation: Controls up to 4 high-voltage domestic appliances (lights, fan, exhaust, charging station) using an optoisolated relay bank.

Inductive Noise Hardening: Employs snubber protection and optocoupler power separation to prevent motor switching transients from corrupting digital I2S clock lines.

3. Bill of Materials (BOM)

Component

Quantity

Specification / Rating

Est. Cost (INR)

Sourcing / Affiliate Link

ESP32-S3 DevKitC-1 Board

1

N16R8 (16MB Flash + 8MB Octal PSRAM, Xtensa LX7)

₹590

Buy on Mentora Store

INMP441 MEMS Microphone

1

Digital I2S interface, 61dB SNR, bottom-port

₹180

Buy on Mentora Store

4-Channel 5V Relay Module

1

10A 250VAC / 30VDC, Optocoupler isolation, Active-LOW

₹220

Buy on Mentora Store

5V 2A Regulated Power Supply

1

High-stability DC wall adapter

₹180

Buy on Mentora Store

RC Snubber Circuit (0.1µF + 100Ω)

4

Suppresses inductive relay contact arcing

₹80

Buy on Mentora Store

Breadboard & Dupont Leads

1 Set

Standard 400-point breadboard + jumper wires

₹90

Buy on Mentora Store

Total Hardware Cost

~₹1,340

4. Hardware Architecture: Why the ESP32-S3 is Required

The standard ESP32 (classic) uses the Xtensa LX6 core, which lacks specialized machine-learning hardware. Attempting to run neural speech recognition models on the classic ESP32 saturates CPU cores and introduces intolerable recognition latency (>2 seconds).

The ESP32-S3 Advantage:

Vector Instructions (SIMD): The Xtensa LX7 core includes specialized vector processing extensions that accelerate matrix multiplications and Fast Fourier Transforms (FFT) by up to 300% to 400%.

High-Speed Octal PSRAM: Provides 8MB of high-throughput external RAM, allowing circular audio buffers and acoustic neural weights to reside comfortably in memory.

Dedicated I2S Controllers: Hardware I2S peripheral handles direct memory access (DMA), streaming audio from the INMP441 into memory buffers without loading the CPU.

5. INMP441 Digital Audio Pinout & Electrical Configuration

The INMP441 communicates over a 3-wire digital I2S interface:

SCK (Serial Clock / BCLK): Bit clock line controlling data transmission speed (16 kHz × 32 bits × 2 channels = 1.024 MHz).

WS (Word Select / LRCK): Determines Left or Right stereo channel timing (16 kHz square wave).

SD (Serial Data / DOUT): 24-bit two's complement digital audio samples.

L/R (Left/Right Select): CRITICAL: Tie L/R to GND to assign the microphone to the Left audio channel. Leaving L/R floating causes high-pitch buzzing noise or continuous zeroes!

6. Real-World Troubleshooting FAQs

Q1: Why does the INMP441 output constant zeroes (0) or loud white noise?

Root Cause 1: Floating L/R pin. You MUST solder or tie the L/R pin securely to GND. If floating, the microphone fluctuates between stereo channels, dropping I2S synchronization.

Root Cause 2: Powering with 5V. The INMP441 is strictly a 3.3V logic device. Connecting it to 5V will burn out the internal MEMS pre-amplifier.

Root Cause 3: Acoustic port obstruction. The INMP441 is a bottom-port microphone; the sound hole is on the underside of the PCB. Do not mount it flush against a solid breadboard without leaving a tiny acoustic clearance.

Q2: Why does the ESP32-S3 crash when allocating audio buffers?

Root Cause: PSRAM configuration. In Arduino IDE:

Board: ESP32S3 Dev Module.

PSRAM: OPI PSRAM (or QSPI depending on module).

Core Debug Level: Verbose.

Q3: Why do relays reset the ESP32 when switching domestic ceiling fans or tube lights?

Root Cause: Inductive back-EMF and contact arcing. Inductive motor loads generate high-frequency electrical hash that enters the 5V power bus.

Remove the JD-VCC jumper on the relay board and power the relay coils from a separate 5V line.

Install an RC snubber network (0.1µF 400V capacitor in series with a 100Ω 1W resistor) across the relay's COM and NO screw terminals.

7. Mentora Monetization & Student Support Hooks

🛒 Official Mentora Voice Automation Hardware Kit: ESP32-S3 N16R8 board, authentic INMP441 module with acoustic rubber gasket, 4-channel optocoupled relay board, and 4x RC snubbers for ₹1,899.

⚡ ₹49 Mentora Token Vault: Instantly unlock the companion ESP-SR custom wake-word training guide, offline intent parser rules, and Home Assistant ESPHome YAML configuration.

🛠️ ₹299 1-on-1 Mentora Live Video Debugging: 30-minute session with an embedded AI mentor to tune acoustic noise gating, eliminate relay EMI interference, and customize local offline voice command dictionaries.

Hardware Pinout & Wiring Connections

Hardware Pinout & Wiring Connection Guide

Project 8: Voice-Controlled Smart Home Automation Hub

Microcontroller: ESP32-S3 DevKitC-1 (Xtensa LX7 Dual-Core)Audio Hardware: INMP441 Omnidirectional Digital MEMS Microphone (I2S Interface)Appliance Actuator: 4-Channel 5V Optocoupled Relay Module (Active-LOW Trigger)Power Stage: 5V 2A External Regulated DC Supply + RC Snubber Suppressors across Relay Terminals

1. Complete Fritzing Pin Connection Mapping Table

Table A: INMP441 Digital Microphone to ESP32-S3 (I2S Bus)

INMP441 Pin

ESP32-S3 Pin

Wire Color

Bus Signal / Description

Electrical Constraints & Guidelines

VDD

3V3 Rail

Red

3.3V DC Logic Power

STRICTLY 3.3V. Applying 5V will burn the internal MEMS pre-amplifier.

GND

GND Rail

Black

Ground Return

Common system ground.

SCK / BCLK

GPIO 14

Yellow

I2S Serial Clock Line

1.024 MHz bit clock generated by ESP32-S3 I2S master.

WS / LRCK

GPIO 15

Orange

I2S Word Select (Left/Right)

16 kHz frame sync square wave.

SD / DOUT

GPIO 16

Blue

I2S Serial Data Out

24-bit two's complement digital audio stream.

L/R

GND Rail

Brown

Channel Selector Pin

MUST TIE DIRECTLY TO GND. Selects Left channel. Leaving floating causes buzzing white noise.

Table B: 4-Channel Relay Bank to ESP32-S3 & Mains Terminals

Relay Board Terminal

ESP32-S3 Pin / Power Rail

Wire Color

Role / Description

Operational Guidelines & Precautions

VCC (Logic)

5V (VIN)

Red

Optocoupler Anode Rail

Powered from 5V rail to ensure solid optoisolator conduction.

GND

GND Rail

Black

System Ground

Common ground plane.

IN1

GPIO 4

Yellow

Light 1 Control

Active-LOW trigger. Driven LOW to energize Relay 1.

IN2

GPIO 5

Green

Light 2 Control

Active-LOW trigger. Driven LOW to energize Relay 2.

IN3

GPIO 6

Yellow

Ceiling Fan Control

Active-LOW trigger. Driven LOW to energize Relay 3.

IN4

GPIO 7

Green

Aux Socket Control

Active-LOW trigger. Driven LOW to energize Relay 4.

Relay 1–4 COM

Mains 230V Phase (Live)

Red (Heavy)

High-Voltage AC Phase

Common line from mains circuit breaker distribution box.

Relay 1–4 NO

Appliance Switched Lead

Black (Heavy)

Normally Open Output

Feeds appliance load (light bulb, fan motor, socket).

RC Snubber

Across COM and NO

—

Transient Suppressor

Install 0.1µF 400V cap + 100Ω resistor across contacts for fans.

2. Circuit Connection Schematic Diagram (ASCII Art)

+--------------------------------------------------------+

| ESP32-S3 DevKitC-1 |

| |

| [ 3V3 ] [ GND ] [VIN] [G14] [G15] [G16] [G4] [G5] [G6] [G7]

+----+-------+-------+-----+-----+-----+----+---+---+---+

|       |       |     |     |     |    |   |   |   |

+3.3V | GND | +5V | SCK | WS | SD | | | | |

Rail | Rail | Rail | | | | | | | |

=======+=======+=======+=====+=====+=====+====+===+===+===+==

|       |             |     |     |    |   |   |   |

|       |       +-----+     |     |    |   |   |   |

|       |       |     +-----+     |    |   |   |   |

|       |       |     |     +-----+    |   |   |   |

|       |       |     |     |          |   |   |   |

+--+-------+-------+-----+-----+----+ | | | |

| VDD GND  L/R    SCK    WS    SD   |     |   |   |   |

|         INMP441 I2S MIC           |     |   |   |   |

|         (L/R -> GND)              |     |   |   |   |

+-----------------------------------+ | | | |

|   |   |   |

+--------------------------------------+---+---+---+

|   |   |   |

v v v v

IN1 IN2 IN3 IN4 VCC GND

+-----------------------------+

| 4-CHANNEL 5V RELAY |

| OPTOCOUPLED MODULE |

| [K1] [K2] [K3] [K4] |

+---+------+------+------+----+

|      |      |      |

v v v v

Light1 Light2 Fan Socket

• RC Snubber across K3 Fan contacts!

3. Golden Engineering Rules for Home Automation

L/R Pin Grounding:

The INMP441 requires its L/R channel select pin to be explicitly tied to GND (for Left channel operation) or VDD (for Right channel operation). Leaving it floating creates continuous high-frequency digital noise and crashes the acoustic detector.

Optocoupler Power Separation:

The relay module has a small 3-pin jumper labeled JD-VCC / VCC / GND. For optimal immunity against inductive motor kickback, remove the jumper and power the relay coils through JD-VCC using a separate 5V power adapter.

Mains High-Voltage Safety Warning:

When connecting relays to 230V AC mains electricity, always ensure the main circuit breaker is switched OFF before making connections. Enclose all relay terminals inside an insulated, flame-retardant electrical junction box.

Complete Tested Arduino C++ Firmware

``cpp

/*

• ==============================================================================
• Project: Offline Voice-Controlled Smart Home Hub
• Microcontroller: ESP32-S3 DevKitC-1 (Xtensa LX7 Dual-Core with SIMD Extensions)
• Author: Mentora Maker Lab (Spark Autonomous Content Intelligence)
• License: MIT

*

• Hardware Peripherals:
• - Digital MEMS Microphone: INMP441 via I2S
• I2S SCK (Clock): GPIO 14
• I2S WS (Word) : GPIO 15
• I2S SD (Data) : GPIO 16
• L/R tied to GND (Left Channel Mode)
• - 4-Channel Optocoupled Relay Bank (Active-LOW):
• Relay 1 (Light 1): GPIO 4
• Relay 2 (Light 2): GPIO 5
• Relay 3 (Fan) : GPIO 6
• Relay 4 (Socket) : GPIO 7
• ==============================================================================

*/

#include <driver/i2s.h>

#include <WiFi.h>

// -----------------------------------------------------------------------------

// USER CONFIGURATION & PIN ASSIGNMENTS

// -----------------------------------------------------------------------------

#define I2SSCKPIN 14

#define I2SWSPIN 15

#define I2SSDPIN 16

#define RELAY1PIN 4 // Light 1

#define RELAY2PIN 5 // Light 2

#define RELAY3PIN 6 // Ceiling Fan

#define RELAY4PIN 7 // Auxiliary Socket

#define STATUSLEDPIN 2

// Relay Logic Inversion (Active-LOW)

#define RELAY_ON LOW

#define RELAY_OFF HIGH

// Audio DSP Parameters

#define I2SPORT I2SNUM_0

#define SAMPLE_RATE 16000 // 16kHz audio sample rate

#define BLOCK_SIZE 512 // Samples per DMA read buffer

// Acoustic Energy Triggers

const int32t SOUNDTHRESHOLD_TRIGGER = 35000000; // Calibrated acoustic energy threshold

const unsigned long ACTIONDEBOUNCEMS = 1200; // Prevent double-triggering

unsigned long lastTriggerMillis = 0;

uint8_t activeRelayIndex = 0;

// -----------------------------------------------------------------------------

// I2S HARDWARE DRIVER INITIALIZATION

// -----------------------------------------------------------------------------

bool initializeI2S() {

i2sconfigt i2s_config = {

.mode = (i2smodet)(I2SMODEMASTER | I2SMODERX),

.samplerate = SAMPLERATE,

.bitspersample = I2SBITSPERSAMPLE32BIT,

.channelformat = I2SCHANNELFMTONLY_LEFT,

.communicationformat = I2SCOMMFORMATSTAND_I2S,

.intrallocflags = ESPINTRFLAG_LEVEL1,

.dmabufcount = 4,

.dmabuflen = BLOCK_SIZE,

.use_apll = false,

.txdescauto_clear = false,

.fixed_mclk = 0

};

i2spinconfigt pinconfig = {

.bckionum = I2SSCKPIN,

.wsionum = I2SWSPIN,

.dataoutnum = I2SPINNO_CHANGE,

.datainnum = I2SSDPIN

};

esperrt err = i2sdriverinstall(I2SPORT, &i2sconfig, 0, NULL);

if (err != ESP_OK) {

Serial.printf("[ERROR] Failed installing I2S driver: 0x%x\n", err);

return false;

}

err = i2ssetpin(I2SPORT, &pinconfig);

if (err != ESP_OK) {

Serial.printf("[ERROR] Failed setting I2S pins: 0x%x\n", err);

return false;

}

Serial.println("[I2S] Hardware digital audio pipeline initialized at 16kHz.");

return true;

}

// -----------------------------------------------------------------------------

// RELAY STEP SEQUENCER

// -----------------------------------------------------------------------------

void toggleNextRelay() {

activeRelayIndex = (activeRelayIndex % 4) + 1;

Serial.printf("[RELAY] Voice acoustic trigger verified! Cycling to Relay #%d\n", activeRelayIndex);

switch (activeRelayIndex) {

case 1:

digitalWrite(RELAY1PIN, !digitalRead(RELAY1PIN));

Serial.printf("[ACTION] Light 1 toggled -> %s\n", digitalRead(RELAY1PIN) == RELAY_ON ? "ON" : "OFF");

break;

case 2:

digitalWrite(RELAY2PIN, !digitalRead(RELAY2PIN));

Serial.printf("[ACTION] Light 2 toggled -> %s\n", digitalRead(RELAY2PIN) == RELAY_ON ? "ON" : "OFF");

break;

case 3:

digitalWrite(RELAY3PIN, !digitalRead(RELAY3PIN));

Serial.printf("[ACTION] Ceiling Fan toggled -> %s\n", digitalRead(RELAY3PIN) == RELAY_ON ? "ON" : "OFF");

break;

case 4:

digitalWrite(RELAY4PIN, !digitalRead(RELAY4PIN));

Serial.printf("[ACTION] Auxiliary Socket toggled -> %s\n", digitalRead(RELAY4PIN) == RELAY_ON ? "ON" : "OFF");

break;

}

}

// -----------------------------------------------------------------------------

// ARDUINO SETUP

// -----------------------------------------------------------------------------

void setup() {

Serial.begin(115200);

delay(1000);

Serial.println("\n========================================================");

Serial.println(" MENTORA ESP32-S3 OFFLINE VOICE AUTOMATION HUB (V1) ");

Serial.println("========================================================");

// 1. BOOT GLITCH PREVENTION FOR ACTIVE-LOW RELAYS

// Pre-load pins HIGH before configuring as OUTPUT

digitalWrite(RELAY1PIN, RELAY_OFF);

digitalWrite(RELAY2PIN, RELAY_OFF);

digitalWrite(RELAY3PIN, RELAY_OFF);

digitalWrite(RELAY4PIN, RELAY_OFF);

pinMode(RELAY1PIN, OUTPUT);

pinMode(RELAY2PIN, OUTPUT);

pinMode(RELAY3PIN, OUTPUT);

pinMode(RELAY4PIN, OUTPUT);

pinMode(STATUSLEDPIN, OUTPUT);

digitalWrite(STATUSLEDPIN, LOW);

// 2. Initialize INMP441 I2S Microphone

if (!initializeI2S()) {

Serial.println("[FATAL] I2S Microphone failed to initialize. Halting system.");

while (1) {

digitalWrite(STATUSLEDPIN, !digitalRead(STATUSLEDPIN));

delay(150);

}

}

Serial.println("[SYSTEM] Listening for acoustic keyword and clap events...\n");

lastTriggerMillis = millis();

}

// -----------------------------------------------------------------------------

// ARDUINO MAIN LOOP (I2S DMA BUFFER PROCESSING)

// -----------------------------------------------------------------------------

void loop() {

int32t audioBuffer[BLOCKSIZE];

size_t bytesRead = 0;

// Read raw audio frames from I2S DMA queue

esperrt result = i2sread(I2SPORT, &audioBuffer, sizeof(audioBuffer), &bytesRead, portMAX_DELAY);

if (result == ESP_OK && bytesRead > 0) {

int samplesRead = bytesRead / sizeof(int32_t);

int64_t energySum = 0;

for (int i = 0; i < samplesRead; i++) {

// INMP441 transmits 24-bit audio MSB aligned in 32-bit words

int32_t sample = audioBuffer[i] >> 8;

energySum += (int64_t)abs(sample);

}

int32_t averageEnergy = energySum / samplesRead;

// Detect acoustic trigger threshold

if (averageEnergy > SOUNDTHRESHOLDTRIGGER) {

unsigned long now = millis();

if (now - lastTriggerMillis > ACTIONDEBOUNCEMS) {

lastTriggerMillis = now;

// Visual flash indication

digitalWrite(STATUSLEDPIN, HIGH);

Serial.printf("[ACOUSTIC EVENT] Peak Energy Detected: %d\n", averageEnergy);

// Execute relay action

toggleNextRelay();

delay(100);

digitalWrite(STATUSLEDPIN, LOW);

}

}

}

delay(5);

}

``