Jwen
Jwen draws faces the way I doodle them in Apple Notes. It is Qwen 2.5 3B, fully fine-tuned to write Apple Notes drawings as pen strokes, then preference-tuned on 660 of my own yes/no verdicts. It predicts pen tool, color, position and timing token by token, so a drawing comes out stroke by stroke in the order a hand would make it.
The live stream, and the write-up of how it was made, are at joshuawolk.com/jwen.
samples/example.json is that drawing's stroke file: 69 strokes, one blue pen, character group 2, both temperatures at 0.9. It is one of the thousand in the live stream, re-rendered here from its tokens.
Eight more from the same thousand, across pen, pencil, marker and crayon. Not every sample is this clean, see Limitations.
Built with Qwen. Released under the Qwen Research License: research and non-commercial use only.
What is in the repo
| File | Purpose |
|---|---|
model-*.safetensors, config.json | the weights, bf16, a standard Qwen2ForCausalLM with the embedding matrix widened to 157,668 rows |
stroke_vocab.json | the 5,732 stroke tokens, in id order |
meta.json | old, the id of the first stroke token (151,936), and the format version |
tokenizer.json, tokenizer_config.json | the base Qwen tokenizer, for the text rows; the stroke tokens are not in it |
jwen/sample.py | grammar-constrained sampler that writes stroke JSON and a rough preview PNG |
jwen/stroke_tokens.py | the token format, grammar and stroke decoder |
pkrender/main.swift, pkrender/Info.plist | PencilKit renderer for macOS, the only way to see the real ink; the plist gives the process the bundle identity PencilKit expects |
samples/ | the example drawing's stroke file, and the two images above |
Quick start
pip install torch transformers pillow
python jwen/sample.py --count 4 --group 3 --out samples
That writes samples/000.json (strokes), samples/000.tokens.txt and samples/000.png (a Pillow polyline preview). Sampling a face is a few thousand tokens; expect well under a minute per drawing on an H100 and three to four minutes on an Apple Silicon Mac with 16 GB or more.
Options: --group 0..82 picks a character group (the model chooses when omitted, but naming one helps), --palette mono|color and --inks one|many steer the look, --temp-struct and --temp-points set separate temperatures for the plan and the points (0.75 and 0.9 by default), --seed fixes the draw.
Rendering the real ink
Strokes carry Apple pencil, pen, marker, crayon, watercolor and fountain pen tools with widths and pressure, and only PencilKit draws those the way Notes does. On macOS:
cd pkrender
xcrun --sdk macosx swiftc -O main.swift -o pkrender -framework PencilKit -framework AppKit \
-Xlinker -sectcreate -Xlinker __TEXT -Xlinker __info_plist -Xlinker Info.plist
./pkrender ../samples/000.json ../samples/000.pk.png 1080
Pass --pkrender pkrender/pkrender to sample.py to render every sample as it lands. --dark on pkrender renders the dark-mode ink.
Token format
One drawing is one sequence:
<face> <cluster:N|none> <type:face> <side:S> <palette:mono|color> <inks:one|many> <count:N>
<plan> { <s> <tool:T> <color:C> <x:X> <y:Y> <ex:X> <ey:Y> <bw:W> <bh:H> }* </plan>
{ <s> <gap:G> <dt:D> <tool:T> <color:C> <x:X> <y:Y> <ex:X> <ey:Y> <bw:W> <bh:H> <w:K> { [<w:K>] (<dx:D> <dy:D> | <x:X> <y:Y>) }* }*
</face>
The header names the character group, palette, ink count and stroke count. The plan lists every stroke's tool, color, start, end and bounding box before any point is drawn. Then each stroke repeats its header and gives its points, thinned to one every 24 ms, as deltas on a 1024 grid. Stroke token i in stroke_vocab.json has model id 151936 + i. jwen/sample.py masks illegal tokens at every step, copies each stroke header from the plan, and stops after the last planned stroke, so free-running generation is not needed and would produce malformed sequences.
Training
- Data: 1,296 of my own Notes doodles, decoded from PencilKit bundles, with a PencilKit re-render checked against the PNG Notes saved. Recurring characters were clustered with DINOv2 into 83 groups, which is where
<cluster:N>comes from. - Supervised: full fine-tune of Qwen 2.5 3B on an H100, 20 passes, weight decay 0.1, attention dropout 0.1. Pass 16 is the reference model.
- Preference: DPO on 660 verdicts over four rounds (beta 2, learning rate 3e-6, 8 pairs per step, per-token averaged log-probabilities, fp32 reference), with a likelihood term on real drawings at weight 0.2 mixed into every step to stop the drift plain DPO causes. This checkpoint is step 100 of that run, chosen over step 200 in a blind 40-pair comparison.
Total GPU time was about $103.
Limitations
It draws faces and little else, in one person's hand. Feature placement is loose: eyes, nose and mouth land in plausible order more often than in plausible places. Sampling without the grammar produces broken sequences. The preview renderer is a stand-in; the ink only looks right through PencilKit.