StreamSonic v5.4 — Base (v4loud)
On-device streaming audio perception — silent by default, speak once at a salient event, don't repeat, correctly alert on danger. 端侧流式音频感知 —— 平时沉默、显著事件处开口一次、不重复念、对危险正确告警。
What this repo is / 这个仓库是什么
This is the base model of StreamSonic v5.4: Qwen2.5-Omni-3B, Thinker only, fine-tuned to the v4loud stage (streaming, 1.5s chunk, "always-describe"). Weights are merged (ready to load). 这是 StreamSonic v5.4 的基座模型：Qwen2.5-Omni-3B，仅 Thinker，微调到 v4loud 阶段（流式、1.5s chunk、随时会描述）。权重已 merged，可直接加载。
Full v5.4 system / 完整系统
v5.4 = this base + two small heads (hosted on GitHub): v5.4 = 本基座 + 两个小头（放在 GitHub）：
| Part 部件 | Where 位置 | Size |
|---|---|---|
| Base 底座 (v4loud, frozen) | this repo 本仓库 | ~8.8 GB |
| Main decision head 主决策头 | GitHub: weights/head.pt | 4.1 M |
| Secondary suppressor 二级抑制头 | GitHub: weights/sup.pt | 0.9 K |
Code & slides / 代码与幻灯片: https://github.com/Buddy-svg/streamsonic
Inference / 推理
python src/train/s2_infer.py --audio <clip>.wav \
--base BUDDY-0912/streamsonic-v5.4-base \
--head weights/head.pt --sup weights/sup.pt \
--sup-thr 0.70 --thr 0.75
Runtime knobs / 运行时旋钮: thr (main head 主头), sup-thr=0.70 (suppressor 抑制强度).
Method (short) / 方法简述
- perceive → respond → decide. Self-built tri-state labeling (positive / negative / ignore) grounded on real acoustic change verified by three detectors (AST / CLAP / JSD) + source-clip review.
- Respond: teach the model to always describe each 1.5s chunk (Projector + LLM LoRA + control-token embeddings; audio tower frozen).
- Decide: a frozen-backbone decision head + a 0.9K secondary suppressor that halves repeated narration.
- 自建三态标注（正 / 负 / ignore），建立在被三检测器投票 + 源片段审查验证过的真实声学变化上；先教"随时会描述"，再学"何时开口 / 不重复"。
Author: Wenbo Gao (Intern) · Manager: Yuzhou Liu · Mentor: Gordan Han · Week 1 → Week 12 (v5.4).