CID-v1-0.4B

CID-v1-0.4B is the 398.8M-parameter compact reference checkpoint for Continuous Interaction Diffusion (CID), a diffusion-native runtime/model architecture for persistent cognitive state, asynchronous tool interaction, and revisable generation. The published weights package the Stage B-updated LFM2.5 diffusion backbone together with the CID-specific modules and frozen semantic encoder snapshot.

This checkpoint is intentionally mechanism-focused: it tests whether CID behavior can be learned at sub-billion-parameter scale. General-purpose chat, coding, and academic benchmark performance are secondary at this model size.

Highlights

On the 512-example held-out CID neural-contract validation set, free rollout reaches:

These results support the feasibility of learning the core CID control structure—especially convergence, cognitive-cell lifecycle behavior, and source selection—with a compact diffusion backbone under free rollout. The checkpoint is best treated as a mechanism and runtime research model whose main value is CID behavior at sub-billion-parameter scale.

A separate public-benchmark sweep covers 44,376 examples across eight datasets with 128/128 shards completed and zero shard failures. Standard benchmark results and failure analysis are reported below as capability diagnostics.

Files

• model.safetensors — the complete CID checkpoint: Stage B-updated LFM2.5 backbone, CID-specific modules, and the frozen semantic embedding snapshot.
• config.json — the unified CIDModel configuration, including nested backbone/CID geometry and Hugging Face AutoModel mappings.
• configuration_cid.py and modeling_cid.py — thin Hugging Face remote-code entry points for the CID implementation.
• cid_config.json — training provenance, hashes, validation metrics, and release metadata.
• tokenizer files — the tokenizer and chat template inherited from the LFM2.5 backbone.
• logs/ — sanitized Stage A / final Stage B training and validation logs.

The release contains 398,775,265 model parameters: 354,483,968 backbone parameters and 44,291,297 CID-specific parameters. model.safetensors additionally carries the frozen BF16 semantic embedding snapshot used by the CID runtime. The trainable/model parameters remain FP32 and no quantization was applied.

Usage

This is not a standalone autoregressive/chat checkpoint. CID inference requires the CID runtime. The backbone, CID modules, and frozen semantic snapshot are all stored in the single model.safetensors file.

Code: https://github.com/fwerkor/continuous-interaction-diffusion

For this unified release format, use code commit beb84cc2831fb547d2744dc50503d7349246c5da or later.

from transformers import AutoModel, AutoTokenizer

model_id = "fwerkor/CID-v1-0.4B"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModel.from_pretrained(model_id, trust_remote_code=True)

Install the CID implementation from the code repository before using the Hugging Face remote-code entry points. The CID runtime can also load the complete package directly with load_cid_model_from_pretrained().

Training data

Training uses fwerkor/CID-Dataset, file release/training-trajectories.jsonl, SHA-256:

07662203cc23f5ee628623090ad029740e51b3d6efb13466a6dcad23a2a3b143

The materialized training corpus contains 192,729 semantic tasks, 422,230 runtime trajectories, 2,724,556 adjacent trajectory transitions, and 3,146,786 total training transitions when causal bootstrap transitions are included. The maximum trajectory length is 44 steps and the maximum TCT capacity is 128 slots.

The neural-contract validation split contains 512 held-out examples / 3,449 validation transitions. Validation SHA-256:

726d864a198302133f9e6005cc14764347a3132f48d5675723b5643d52b754dc

Dataset DOI: 10.57967/hf/10418.

Training

Training used 4× NVIDIA RTX A6000 48 GB GPUs with BF16 mixed-precision compute.

Stage A — CID structure learning

Stage A freezes the backbone and trains the CID-specific modules. The final Stage B run starts from the established Stage A epoch-3 handoff.

Stage B — final full-model adaptation

Stage B starts from the Stage A epoch-3 handoff, unfreezes the backbone, and jointly adapts the backbone and CID modules. Training uses a balanced sampler and a WSD-style learning-rate schedule.

The Stage B training commit is 68a024fbdd3fd885aac3fd1c8e15e47dc35b4130. The reused Stage A handoff was produced from the earlier training snapshot 6e1e327d8777c0d5d1bb622dc1f848f7a3356959.

CID mechanism validation

The final checkpoint is evaluated on the 512-example held-out validation split with fixed noise seed 1000003 under both teacher-forced and free-rollout evaluation. Free rollout is the primary mechanism result because it exercises the model/runtime loop without teacher-forced state updates.

Public benchmark diagnostics

We also ran a full public sweep on ARC-Challenge, MMLU, GSM8K, MATH, MBPP, HotpotQA, 2WikiMultiHopQA, and MuSiQue. These tasks were not the optimization target of CID training and are reported to characterize what the compact checkpoint can and cannot do. The sweep used benchmark code commit 5577a801c9927062989179eb45cd8f6ec7f3a191, max_steps=64, and completed all 44,376 examples without a failed shard. Tool-enabled tasks use the deterministic calculator, Python, symbolic-math, or task-local workspace interfaces defined by the benchmark, with no injected runtime-step delay.

The strict multiple-choice scorer only accepts an exact generated option string or label. ARC-Challenge and MMLU therefore remain sensitive to Display realization as well as underlying task accuracy.

The public sweep exposes several scale-dependent limitations:

• Multiple-choice Display realization. A substantial fraction of ARC/MMLU outputs express truncated, misspelled, or paraphrased option text that fails strict option matching; genuine knowledge/reasoning errors also remain.
• Mathematical reasoning. GSM8K and MATH failures are mostly well-formed but incorrect numeric/formula outputs, consistent with limited general reasoning capacity at this scale. Malformed text is comparatively uncommon.
• Code generation. MBPP failures are dominated by invalid or incomplete programs (including syntax errors and NameErrors); this checkpoint was not code-specialized.
• Long-horizon tool control. HotpotQA, 2WikiMultiHopQA, and MuSiQue still frequently exhaust the 64-step compute budget before stabilizing a final Display. With synthetic tool delay removed, the three retrieval tasks improve in F1 relative to the earlier sweep, while repeated information needs and unstable bindings remain the dominant bottleneck at 0.4B scale.

Together, these diagnostics separate CID mechanism acquisition from end-task capacity: the held-out neural-contract evaluation measures CID-specific control behavior, while the public tasks expose the current limits of the 0.4B backbone/runtime interaction. Larger backbones are the next scaling test.

Training logs

Sanitized release logs are under logs/:

• logs/stage-a/train_metrics.rank-0000.jsonl
• logs/stage-a/validation_metrics.jsonl
• logs/stage-b/train_metrics.rank-0000.jsonl
• logs/stage-b/validation_metrics.jsonl
• logs/run_config.json
• logs/manifest.json

The public logs preserve numerical metrics, timestamps, optimizer steps, learning rates, losses, validation results, and Git commit identifiers while redacting local usernames, absolute paths, internal host names, and local job paths.

Checkpoint provenance

• Base model: LiquidAI/LFM2.5-Encoder-350M-Diffusion
• LR schedule: WSD-linear
• Optimizer steps: 96,801
• Stage B training transitions seen: 803,810
• Code commit: 68a024fbdd3fd885aac3fd1c8e15e47dc35b4130

The source training checkpoint used 4-rank FSDP storage. The published package removes optimizer state and rank-local training state and converts the model into portable inference weights.

Intended use and limitations

This checkpoint is intended for CID research, reproduction, runtime development, and small-scale mechanism studies. Its primary evidence is the acquisition of CID-specific runtime behavior at 398.8M parameters. General chat, mathematics, coding, and long-horizon multi-hop performance remain capacity-limited at this scale. End-to-end evaluation uses the CID runtime, which supplies the state-update semantics required by the checkpoint.

Paper and citation

This checkpoint accompanies the CID paper: Continuous Interaction Diffusion: A Diffusion-Native Architecture for Asynchronous Tool-Augmented Reasoning (arXiv:2608.10438, DOI).

If you use this model or the CID runtime, please cite:

@article{cao2026continuous,
  title   = {Continuous Interaction Diffusion: A Diffusion-Native Architecture for Asynchronous Tool-Augmented Reasoning},
  author  = {Cao, Yuhang and Mu, Yanzhou and Fang, Chunrong and Chen, Zhenyu},
  journal = {arXiv preprint arXiv:2608.10438},
  year    = {2026},
  doi     = {10.48550/arXiv.2608.10438},
  url     = {https://arxiv.org/abs/2608.10438}
}

License

The backbone is derived from Liquid AI's LFM2.5-Encoder-350M-Diffusion and is distributed under the accompanying LFM 1.0 license terms in LICENSE.