RalphQwen3.8 Flash Next (tiny Qwen3.8-Flash-Next test fixture)
A 193M-parameter model with the Qwen3.8-Flash-Next architecture. Its weights were randomly initialized, then trained from scratch on a small synthetic corpus. It exists so that Qwen3.8-Flash-Next support can be built and tested without the full model.
This is not a usable language model. It is four layers deep and 320 wide, which is enough to carry the structures of the text model and nothing more. It answers everything with a confident, childlike non-fact, which is what it was trained to do:
I think raindrops put the moon to bed. They sound like the last Tuesday.
Turtles keep tiny houses in their pajamas. The breakfast spoon made me the toast captain!
For use as a fixture, not as a model. It is the Qwen sibling of RalphSeek V4 Flash and RalphSeek V4.1 Flash.
Structure
| property | this model | Qwen3.8-Flash-Next |
|---|---|---|
| architecture | qwen4exp | same |
| layers | 4: 3 linear attention, then 1 full attention | 48: 36 linear attention, 12 full attention |
| hidden size | 320 | 2560 |
| attention heads / KV heads / head dim | 4 / 1 / 128 | 24 / 2 / 256 |
| linear attention key / value heads | 2 / 4 | 16 / 48 |
| routed experts / used | 8 / 2 | 512 / 10 |
| expert and shared expert width | 640 | same |
| hyper-connection streams | 4, low rank 320 | same |
| sparse attention indexer | 2 heads, compress ratio 4, budget 16 | 4 heads, compress ratio 4, top-k 2048 |
| per-layer token embedding | on blk.1, n-gram size 3, 2 hashed heads of 4,099 and 4,111 rows, 160 wide | on blk.1, n-gram size 3, 16 hashed heads of about 20 M rows, 160 wide |
| per-layer token embedding parameters | 1.33 M | 51.2 G |
| vocabulary | 248,320 | same |
| configured context | 4,096 | 262,144 |
| MTP layers | 0 | 1 |
| vision tower | none | present |
| parameters | 193.02 M |
It trained on the RalphSeek corpus of 6,560 training, 816 validation, and 816 test examples. Prompts used the Qwen chat template with thinking disabled, and the loss covered the reply only. Training ran 664 steps, then 2,142 more over two epochs. Final validation loss was 0.441, and test loss was 0.442.
Files
| file | size | use |
|---|---|---|
RalphQwen3.8-Flash-Next-F32.gguf | 783 MB | every tensor F32 |
Usage
llama-cli -m RalphQwen3.8-Flash-Next-F32.gguf -c 1024 -fa on --temp 0 -n 64 -e \
-p "<|im_start|>user\nTeacher: Tell me about raindrops.\nRalph:<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n"
In the corpus a teacher asks and Ralph answers, so a prompt in that shape gets the kind of answer it was trained on.
Other runtimes
The GGUF was converted with the qwen4exp converter in ggml-org/llama.cpp at f8dbcd6. Loading it in upstream llama.cpp is untested. There is no MTP head.
Provenance and licensing
- Weights are original, trained from random initialization on the synthetic corpus used for RalphSeek V4 Flash. They are not derived from, distilled from, or a slice of any Qwen checkpoint. The hidden size alone (320 against 2560) makes that impossible.
- Architecture is the Transformers
Qwen4Exptext model (qwen4_exp_text, Transformers 5.18.0.dev0), with the geometry above scaled down from theconfig.jsonof Qwen/Qwen3.8-Flash-Next. - Tokenizer and chat template are copied from the same repository, so that prompts and special tokens behave as they do on the real model. They are the reason for the license above.
- Not affiliated with or endorsed by Qwen or Alibaba.
The name is the same joke as RalphSeek's, about a small thing that says wrong facts with total confidence.