Firebot Voice Intent Classifier & Speaker Verification Models

Official model checkpoints and speaker verification voiceprints for Firebot, an autonomous tactical firefighting robot platform with dual-tier offline voice intent recognition and personalized acoustic adaptation.

Overview

The Firebot voice system is engineered for zero-latency, high-reliability command recognition in noisy operational environments. It combines:

1. Frozen Whisper Encoder Feature Extraction: 384-dimensional pooled audio embeddings from OpenAI's Whisper model (tiny.en / base.en).
2. L2-SP Regularized MLP Intent Classifier: A lightweight Multi-Layer Perceptron (LayerNorm $\to$ Linear 384$\times$128 $\to$ GELU $\to$ Dropout $\to$ Linear 128$\times$15) fine-tuned with an L2 penalty pulling weights toward the base anchor.
3. ECAPA-TDNN Multi-Clip Acoustic Voiceprints: 192-dimensional speaker embeddings for tactical operator authentication and biometric voice gating.
4. Adaptive Online Calibration: Few-shot per-operator head adaptation enabling rapid personalization from 1–5 takes per command with held-out generalization validation.

Model Architecture & Directory Layout

firebot-voice-intent/
├── intent_head.pt           # Base 15-class Whisper intent classification head
├── intent_head.json         # Architecture configuration, pooling, and class map
├── intent_prototypes.pt     # Latent prototype embeddings for few-shot similarity
├── users/
│   ├── ananya.pt            # Calibrated personal Whisper head for Operator Ananya
│   ├── ananya.json          # Empirical training metrics and per-class take breakdown
│   ├── ananya.history.jsonl # Training iteration telemetry log
│   ├── avinandan.pt         # Calibrated personal Whisper head for Operator Avinandan
│   ├── avinandan.json       # Calibration metrics
│   └── avinandan.history.jsonl
└── voiceprints/
    ├── ananya.npy           # ECAPA-TDNN biometric speaker voiceprint embedding
    └── avinandan.npy        # ECAPA-TDNN biometric speaker voiceprint embedding

Command Vocabulary (15 Closed-Set Classes)

Quickstart: Python Inference

import soundfile as sf
import torch
from huggingface_hub import hf_hub_download
from firebot.voice_intent.infer import IntentClassifier

# 1. Download model from Hugging Face Hub
ckpt_path = hf_hub_download(repo_id="anabaena/firebot-voice-intent", filename="intent_head.pt")

# 2. Instantiate Intent Classifier (loads frozen Whisper + lightweight head)
classifier = IntentClassifier(checkpoint_path=ckpt_path)

# 3. Classify raw audio (16kHz mono WAV)
audio, sr = sf.read("command_sample.wav", dtype="float32")
prediction = classifier.predict(audio, sample_rate=sr)

print(f"Predicted Intent: {prediction['label']}")
print(f"Confidence: {prediction['confidence']:.2%}")
print(f"Action: {prediction['canonical_phrase']}")

Biometric Speaker Verification (ECAPA-TDNN)

import numpy as np

# Load enrolled speaker voiceprint
enrolled = np.load("voiceprints/ananya.npy")

# Compare with live speaker embedding
cosine_sim = np.dot(enrolled, live_embedding) / (np.linalg.norm(enrolled) * np.linalg.norm(live_embedding))
is_authorized = cosine_sim >= 0.72  # calibrated acceptance threshold
print(f"Speaker Verified: {is_authorized} (Similarity: {cosine_sim:.3f})")

Performance & Evaluation

• Inference Latency: ~35ms on CPU (Apple Silicon / modern x86_64)
• Base Intent Accuracy: 94.2% on diverse acoustic validation test set
• Personalized Operator Accuracy: 98.8% – 100.0% with 3–5 takes per command
• Memory Footprint: ~210 KB for the MLP head, ~75 MB for frozen Whisper tiny encoder

License

MIT License. Designed and developed as part of the Firebot Autonomous Robotic Platform.