Parakeet-EOU-120M CoreML INT8
Streaming speech recognition with end-of-utterance detection, converted to CoreML for Apple Neural Engine inference. Part of speech-swift — on-device speech AI for Apple Silicon.
Based on nvidia/parakeet_realtime_eou_120m-v1 (FastConformer-RNNT, cache-aware streaming).
Quick Start
// Add to Package.swift:
// .package(url: "https://github.com/soniqo/speech-swift.git", branch: "main")
import ParakeetStreamingASR
let model = try await ParakeetStreamingASRModel.fromPretrained()
// Streaming transcription from microphone
for await partial in model.transcribeStream(audioChunks: micStream) {
print(partial.text, partial.isFinal ? "[FINAL]" : "")
}
Or via CLI:
git clone https://github.com/soniqo/speech-swift && cd speech-swift && make build
.build/release/audio transcribe --engine parakeet-eou --stream recording.wav
Model
| Property | Value |
|---|---|
| Parameters | 120M |
| Architecture | FastConformer-RNNT (17-layer encoder, 1-layer LSTM decoder) |
| Format | CoreML (.mlmodelc) |
| Quantization | INT8 palettization (encoder) |
| Vocabulary | 1024 BPE + EOU + EOB + blank (1027 total) |
| Sample rate | 16 kHz |
| Streaming chunk | 320ms (configurable) |
Files
| File | Size | Description |
|---|---|---|
encoder.mlmodelc | 102 MB | Cache-aware FastConformer encoder (INT8) |
decoder.mlmodelc | 7.5 MB | 1-layer LSTM prediction network |
joint.mlmodelc | 2.7 MB | RNNT joint network (1027 outputs) |
config.json | <1 KB | Model configuration and streaming params |
vocab.json | <1 KB | BPE vocabulary (1026 tokens) |
Performance
Benchmarked on Apple M2 Max, 320ms chunks, CoreML CPU+NE compute units.
| Metric | Value |
|---|---|
| Encoder latency (mean) | 17.0 ms |
| Full pipeline latency (mean) | 17.7 ms |
| Full pipeline latency (P95) | 22.4 ms |
| RTF | 0.055 (18x real-time) |
| Streaming cache | 2.6 MB |
| Total model size | 112.4 MB |
| Peak RSS delta (100 chunks) | 0 MB |
Streaming Cache Shapes
cache_last_channel: [17, 1, 70, 512] float32 2.4 MB
cache_last_time: [17, 1, 512, 8] float32 0.3 MB
cache_last_channel_len: [1] int32
LSTM h: [1, 1, 640] float16
LSTM c: [1, 1, 640] float16
Usage
import ParakeetStreamingASR
let model = try await ParakeetStreamingASRModel.fromPretrained()
for await partial in model.transcribeStream(audioChunks: micStream) {
print(partial.text, partial.isFinal ? "[FINAL]" : "")
}
EOU Detection
End-of-utterance is detected via a special <EOU> token (ID 1024) emitted by the joint network. No external VAD required for utterance segmentation.
Source
Converted from nvidia/parakeet_realtime_eou_120m-v1 using coremltools 8.3 with INT8 palettization.
Links
- speech-swift - Apple SDK
- soniqo.audio - Website
- blog.ivan.digital - Blog
- Guide: soniqo.audio/guides/dictate
- Docs: soniqo.audio
- GitHub: soniqo/speech-swift