plaidq-0.7b-16step

A PlaidQ checkpoint: a continuous (Gaussian) latent-diffusion language model. Instead of predicting tokens left to right, PlaidQ diffuses a whole sequence in a 16-dimensional continuous token-embedding latent and denoises it with a bidirectional Qwen3-0.6B trunk loaded verbatim from the pretrained autoregressive checkpoint, under a learned monotone noise schedule. Generation is therefore parallel over positions, and the number of denoising steps is a free knob: the same model can be sampled for hundreds of steps or, after distillation, for a handful.

Because the trunk is initialized from Qwen/Qwen3-0.6B, this model is a derivative of it and is released under the same Apache-2.0 license.

Weights only: 383 tensors, 717.4M parameters, fp32, bit-identical to the trained master weights.

What this checkpoint is

Training recipe

Usage

This is not a transformers architecture. It needs the PlaidQ repository, which supplies the denoiser, the noise schedule and the reverse-diffusion sampler:

git clone https://github.com/pengzhangzhi/plaidq && cd plaidq
pip install -r requirements.txt

Sample from the hub spec directly -- every CLI that takes a checkpoint accepts it, and hf://fredzzp/plaidq-0.7b-16step@<branch-or-sha> pins a revision:

python -m plaidq.sample \
    --checkpoint_path=hf://fredzzp/plaidq-0.7b-16step \
    --prompt="def fibonacci(n):" \
    --sampling_timesteps=128 --score_temp=0.5 --n_samples=4

--sampling_timesteps is the compute/quality dial; --score_temp divides the predicted noise and is the single most sensitive sampling knob. From Python:

import plaidq.hub
from plaidq.eval.loader import load_eval_model

em = load_eval_model("hf://fredzzp/plaidq-0.7b-16step", device="cuda")            # eval bundle
payload = plaidq.hub.load_checkpoint_payload("hf://fredzzp/plaidq-0.7b-16step")   # raw module states

Evaluation

Benchmarks are run with the repository's own harness, offline against locally staged data (python scripts/download_eval_tasks.py):

python -m plaidq.eval --ckpt=hf://fredzzp/plaidq-0.7b-16step --config_json=<an eval config from configs/>

Protocol for the code benchmarks: generation is zero-shot, with a 128-token response budget on a min(|prompt| + 128, 2048)-token canvas; the prompt occupies a clean prefix and every remaining position starts as Gaussian noise. HumanEval and HumanEval+ use the dataset prompt verbatim; MBPP and MBPP+ use the problem text plus its three public assertions followed by an opening ```python fence. Completions are truncated at task stop strings, sanitized to the longest span that parses as Python, then executed against the benchmark tests in a fresh process with a 15 s timeout. pass@k uses the unbiased estimator over 20 samples per problem. Infilling (HumanEval-Infill, SantaCoder-FIM) conditions on a clean prefix and suffix and denoises a fixed 64-token gap -- the no-oracle setting.

Reference results

Scores for the released PlaidQ models, for orientation (pass@10 unless noted; an export of a different checkpoint will differ, so re-measure before quoting):

The 16-step student beats its own teacher sampled for 512 steps, and the one-step student still emits functionally correct programs from a single denoising step.

Not included

• Tokenizer. Use Qwen/Qwen3-0.6B; the loader fetches it by name.
• Optimizer / RNG / data-loader state. Inference weights only, so this checkpoint cannot resume training: that state is sharded per rank and is only meaningful to a job with the same number of ranks as the one that wrote it.
• Transformer-Engine _extra_state buffers (28 empty fp8 bookkeeping tensors), dropped on export; the loader tolerates their absence by design.

Citation

@article{plaidq,
  title   = {Distilled Continuous Diffusion Language Models Can Write Code in Few Steps---or One},
  author  = {PlaidQ authors},
  year    = {2026},
}

Provenance

{
  "checkpoint_reason": "milestone",
  "exported_at": "2026-09-04T05:17:01Z",
  "next_step": 4000,
  "source_checkpoint": "milestone_3999.pt"
}