TeamFolklore_Model

Overview & Wake-Word Detection

Our model listens in steps of 30 ms. At every step it takes the three newest frames of log-mel energy, updates what it remembers of the last one and a half seconds, and gives back one number: the chance that someone has just finished saying Utho Kikkar (ਉੱਠੋ ਕਿੱਕਰ · उठो किक्कर), which means "wake up, Kikkar". We run it entirely on the board, and no audio leaves the device until it fires.

We trained it to fire only at the end of the whole phrase. We taught it that Utho on its own, Kikkar on its own, the two the wrong way round, and everyday words that sound close, like chakkar, shakkar or kicker, are all things that are not the wake word.

It is the listener inside Kikkar, our Smart India Hackathon 2026 project. When it fires, a flower blooms on the board's screen and we stream whatever you say next to a speech recogniser in the cloud.

Quickstart & Verification Pipeline

# pip install numpy soundfile tensorflow huggingface_hub
import numpy as np
import soundfile as sf
import tensorflow as tf
from huggingface_hub import hf_hub_download

path = hf_hub_download("SherTheCoder/TeamFolklore_Model", "kikkar_kws_int8.tflite")
interp = tf.lite.Interpreter(model_path=path)
interp.allocate_tensors()
inp, out = interp.get_input_details()[0], interp.get_output_details()[0]

We wrote the front end below to follow our model's specification. Features that are even slightly different will quietly cost accuracy, so if your own pipeline differs in any detail, trust that instead.

SR, WIN, HOP, NFFT, MELS, STEP = 16000, 480, 160, 512, 40, 3

def _mel(hz):
    return 2595.0 * np.log10(1.0 + hz / 700.0)

def _hz(mel):
    return 700.0 * (10.0 ** (mel / 2595.0) - 1.0)

_edges = _hz(np.linspace(_mel(20.0), _mel(7600.0), MELS + 2))
_bins = np.arange(NFFT // 2 + 1) * SR / NFFT
_fbank = np.zeros((MELS, NFFT // 2 + 1))
for b in range(MELS):
    lo, mid, hi = _edges[b:b + 3]
    rise = (_bins > lo) & (_bins < mid)
    fall = (_bins >= mid) & (_bins < hi)
    _fbank[b, rise] = (_bins[rise] - lo) / (mid - lo)
    _fbank[b, fall] = (hi - _bins[fall]) / (hi - mid)
_hann = np.hanning(WIN)

def features(audio, cms_seconds=3.0):
    """16 kHz mono int16 in, 40 normalised log-mel values per 10 ms frame out."""
    x = np.asarray(audio, dtype=np.float64) / 32768.0
    frames = np.array([x[s:s + WIN] * _hann for s in range(0, len(x) - WIN + 1, HOP)])
    power = np.abs(np.fft.rfft(frames, NFFT)) ** 2
    logmel = np.log(np.maximum(power @ _fbank.T, 1e-8))      # floor at -80 dB
    a = 1.0 - np.exp(-(HOP / SR) / cms_seconds)
    mean = logmel[:50].mean(axis=0)                          # start from the first 0.5 s
    out = np.empty_like(logmel)
    for i, frame in enumerate(logmel):
        mean += a * (frame - mean)
        out[i] = frame - mean
    return out

def wake_scores(audio):
    """One probability per 30 ms step, fed to the model the way the board feeds it."""
    feats = features(audio)
    interp.reset_all_variables()                             # fresh streaming state
    scale, zero = inp["quantization"]
    scores = []
    for s in range(0, len(feats) - STEP + 1, STEP):
        x = feats[s:s + STEP]
        if inp["dtype"] == np.int8:
            x = np.clip(np.round(x / scale) + zero, -128, 127)
        interp.set_tensor(inp["index"], x.astype(inp["dtype"]).reshape(inp["shape"]))
        interp.invoke()
        scores.append(float(interp.get_tensor(out["index"]).reshape(-1)[0]))
    return np.array(scores)

Our model ends in a float sigmoid, so each score is already a probability. Then point it at a clip:

audio, sr = sf.read("clip.wav", dtype="int16")
assert sr == SR and audio.ndim == 1, "the model expects 16 kHz mono audio"
scores = wake_scores(audio)
smoothed = np.convolve(scores, np.ones(3) / 3)[:len(scores)]    # the last 3 steps, 90 ms
print("highest smoothed score:", round(float(smoothed.max()), 3))

To see where the board would wake up (on the board we use a threshold of 0.7, picked on the int8 model):

def detect(audio, threshold, refractory_s=1.0):
    """Times, in seconds, at which the board would wake up."""
    scores = wake_scores(audio)
    smoothed = np.convolve(scores, np.ones(3) / 3)[:len(scores)]
    wakes, last = [], -np.inf
    for i, p in enumerate(smoothed):
        t = ((i * STEP + STEP - 1) * HOP + WIN) / SR
        if p >= threshold and t - last >= refractory_s:
            wakes.append(round(t, 2))
            last = t
    return wakes

Audio Frontend & Feature Extraction

We rely on the running mean to soak up the differences between microphones, gain settings and rooms, so we do not need to recalibrate the board for every house. We start it from the first half second of audio after boot.

Model Architecture & Layer Specifications

We built a streaming depthwise-separable CNN in five layers. The first two work across frequency while that axis still exists, learning spectral shapes cheaply. The third folds frequency into a single 48-channel vector. Everything after that is causal depthwise convolution over time plus 1×1 pointwise mixing, which we chose because it streams as a small ring-buffer update at every step and is the int8 path that microcontroller kernels run fastest.

Layer Breakdown & Compute (MACs)

Shapes are per 30 ms step. We make every convolution over time causal, padded on the left only, and every convolution over frequency keeps its size.

We picked dilations of 1, 1, 2, 2, 3, 3, which give the trunk 49 steps of context, 1.47 s, and the stem and the window overhang add about 40 ms more: ≈ 1.51 s in all. We sized that to cover Utho Kikkar, which takes about 1.0 to 1.2 s to say, with room for slow speakers, and no more. Extra context would cost RAM and let the model latch on to background sound.

At 33.3 steps a second our model does 1.53 million multiply-accumulates a second.

RAM Allocation & Tensor Arena Budget

That leaves us about 186 KB of the 256 KB limit for everything else. The peak activation is only 960 bytes, at S1's output; the arena is mostly kernel scratch and alignment.

Ring Buffers & Streaming State

We keep this state between steps, inside the TFLM arena as resource variables, so we allocate nothing separately.

A dilated causal depthwise convolution needs (k − 1) × d × C bytes of history at int8.

Flash

Post-Processing & Trigger Logic

The wake-up comes quickly because of how we labelled the model. We turn its target on one step before the phrase ends, so by the time the step holding the end is done, two of the three steps in the average are already high, and one more step takes it over τ. Computing a step takes a millisecond or less, so it barely adds to the 60 ms.

Quantisation moves the operating point, so we always pick the threshold on the int8 model and never carry it over from the float one. If false activations will not come down, the cheapest remaining lever we have is two or three frames of lookahead, paid for directly in latency.

Training Methodology & Supervision

Phrase-completion supervision. We built the head as a per-step detector, not a clip classifier. We label only the six steps from one before the end of the phrase to four after it, about 180 ms, as 1. We label every other step 0, including the middle of the phrase, and partial or reversed phrases 0 throughout. That is what teaches the model word order, and it makes the firing delay equal to the smoothing window. We get the phrase end from energy-based silence trimming on recordings, or straight from the synthesiser for TTS clips, and when we speed a clip up or slow it down, we move its end by the same factor.

Loss and selection. We use per-step weighted binary cross-entropy. Positives are about 6 steps in 300, but a positive weight of 50 over-fires, so we start it at 8; we weight confusable negatives 8 and easy ones 1. We choose checkpoints on the fewest false activations per hour of held-out continuous speech first, and on recall only among those that meet the false-activation target, never on accuracy.

Positives. We use about 2,000 real recordings, split by speaker so that the model never hears validation and test speakers in training. On top of that, we add several thousand synthetic renditions from AI4Bharat Indic-TTS, IndicF5, Indic Parler-TTS, MMS-TTS in Punjabi and Hindi, and Piper, with varied prosody and gaps between the two words. We oversample the real recordings so they carry 25 to 40% of the positive signal, otherwise the model learns vocoder artefacts.

Confusable negatives.

Background negatives. We use continuous Hindi and Punjabi speech from Common Voice, IndicVoices and Shrutilipi, plus sounds from around the house, and stream them as long segments so that training sees what the board hears.

Augmentation. We apply room impulse responses (OpenSLR-28, BUT ReverbDB) at different distances; noise from MUSAN, DEMAND and ESC-50 at 0 to 20 dB SNR; gain from −20 to 0 dBFS; speed 0.9 to 1.1; pitch ±2 semitones; microphone response tilt; mild clipping; frequency masking. We keep time masking light, because the labels depend on timing.

Hard negatives. We train, stream the model over more than 10 hours of Hindi, Punjabi and English speech, collect every false trigger as a new negative, and train again, two or three times over.

Streaming. We train with causal convolutions over long 8 to 10 s chunks, which is fast and parallel, then deploy the same weights running step by step on ring buffers. We check that the streaming and full-chunk outputs agree within 1e-3 before quantising.

Quantisation. We quantise after training, with a representative set of real streaming features rather than synthetic clips, and fold batch norm into the convolutions before export.

Hardware Profiling & Benchmarks

These are our numbers for the whole application, networking and display included, not the model alone. Of the 80 ms, 60 ms is our model making sure; the other 20 ms is the board getting the first audio to the server over Ethernet. Wi-Fi takes 30 ms for the same step on the ESP32.

Core Utilization & RTOS Tasks

On an ESP32-S3 at 240 MHz with ESP-NN int8 kernels:

We run capture and inference as separate FreeRTOS tasks pinned to different cores. A classic ESP32 has no vector unit and runs the inference about three times slower, which still fits well inside 10% of a core.

Target Firmware Integration (STM32 & ESP32)

On the ESP32 we run the model under TensorFlow Lite for Microcontrollers, with int8 kernels from ESP-NN. Its 2.4 KB of streaming state lives inside the TFLM arena as resource variables, so we have nothing extra to allocate. Size the arena from arena_used_bytes() measured on the device, not from the estimate above.

On the STM32N657 we do not use TFLM at all. We wrote a small exporter that turns this .tflite into a flat list of 23 operations, and our firmware runs them with its own int8 kernels, using Helium vector instructions for the dot products. It keeps the same 2,384 bytes of history in rings, one per layer that looks back in time, and the whole model works inside a 5.3 KB arena. Our firmware, the exporter, the flower and the streaming pipeline are on GitHub.

Field Testing & Edge Constraints

• Test it on real people in real rooms, and pick the threshold there, on the int8 model.
• It knows one phrase. Ours is not a general speech model, and our model cannot currently tell who is speaking.
• The front end has to match. The running-mean normalisation is part of the input, and it needs its first half second after boot to settle.
• Privacy is the point of our design. We keep all audio on the board until the wake word is confirmed.

Licence

We release it under the Apache 2.0 licence.

Citation

@misc{teamfolklore2026kikkarkws,
  title        = {Kikkar KWS: a 44 KB streaming wake-word model for Utho Kikkar},
  author       = {{Team Folklore}},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/SherTheCoder/TeamFolklore_Model}}
}

Made by

We are Team Folklore, and we made this for Smart India Hackathon 2026. Our firmware, flower and streaming pipeline are on GitHub; our training data is SherTheCoder/TeamFolklore_UthoKikkar.