Neodragon — LiteRT (.tflite) conversions for on-device text-to-video
LiteRT conversions of Qualcomm-AI-Research/Neodragon, prepared for Box — an open-source, fully offline on-device AI app for Android.
These are converted artifacts only. The models, research and original weights are Qualcomm AI Research's work; see the upstream repo and paper.
Generates a 320×512 clip entirely on-device — no network, no server, no account.
Licence — please read before use
The weights are released by Qualcomm under two licences, and both carry over to these conversions:
- BSD 3-Clause Clear — permits redistribution and derivative works (a conversion is a derivative) with notices retained. Note the "Clear" variant grants no patent licence.
- Qualcomm Responsible AI License — adds use restrictions. It is not a grant licence; it sits on top of the BSD grant and flows down to you. Prohibited uses include military use, criminal/predictive-policing applications, social scoring, biometric categorisation, emotion recognition in workplaces or education, and distributing generated content without disclosing that it is machine generated. Read the full text before using these files.
Box surfaces these restrictions in a consent screen before download and marks generated output as AI-generated.
What's here
Fixed-shape LiteRT graphs. Filenames are deliberately flat — Box's downloader flattens remote paths into a single directory.
Video pipeline (picture → video)
| file | size | shape |
|---|---|---|
dt5_text_encoder.tflite | 495 MB | (1,128) ids + mask → (1,128,4096) |
context_adapter_dynamic_int8.tflite | 124 MB | (1,128,4096) → (1,128,1536) |
clip_l_text_encoder.tflite | 472 MB | (1,77) → pooled 768 |
clip_g_text_encoder_int8.tflite | 670 MB | (1,77) → pooled 1280 |
dit_stage0_10x16_int4.tflite | 816 MB | pyramid stage 0 |
dit_stage1_20x32_int4.tflite | 817 MB | pyramid stage 1 |
dit_stage2_40x64_int4.tflite | 822 MB | pyramid stage 2 |
causal_video_vae_encoder_320x512.tflite | 419 MB | (1,3,1,320,512) → (1,16,1,40,64) |
causal_video_vae_decoder.tflite | 38 MB | (1,16,2,40,64) → (1,3,9,320,512) |
The three DiT graphs are the same 1.5B weights at three input resolutions; the pipeline calls one per pyramid stage. Each accepts exactly two conditioning latents, which is what limits this set to 9-frame clips.
First-frame pipeline (text → picture, SSD1B at 640×1024)
| file | size | shape |
|---|---|---|
ssd_1b_clip_l.tflite | 443 MB | (1,77) → (1,77,768) hidden_states[-2] |
ssd_1b_clip_g.tflite | 2.65 GB | (1,77) → (1,1280) + (1,77,1280) |
ssd_1b_unet_640x1024.tflite | 5.07 GB | (1,4,80,128) + t + ehs + text_embeds + time_ids |
ssd_1b_vae_decoder_640x1024.tflite | 189 MB | (1,4,80,128) → (1,3,640,1024) |
Tokenizer assets
| file | note |
|---|---|
clip_vocab.json, clip_merges.txt | CLIP BPE, shared by all four CLIP encoders |
t5_spiece.model | DT5 unigram SentencePiece |
clip_merges.txt retains its #version: 0.2 header line — skip it when
loading, or every merge rank shifts by one and tokenization silently corrupts.
Notes for anyone reusing these
- CPU only. The LiteRT GPU delegate supports at most rank-4 tensors, and
Neodragon's latents are 5D
(b,c,t,h,w). On a Tensor G5,--gpuplaces 256 of 3402 ops on the GPU and then segfaults (RESHAPE: Tensor dimensions must be less than 5). This is structural, not a configuration problem. - Two different CLIP pairs.
clip_*are Neodragon's own encoders (for the DiT's pooled vector);ssd_1b_clip_*are SSD1B's (for SDXL conditioning). Their weights differ. The tokenizer vocabularies are identical, so one tokenizer serves both. - SDXL uses
hidden_states[-2], notlast_hidden_state, from both CLIP encoders. Using the last layer does not crash — it quietly degrades output. - The SSD1B UNet is fp32 because int8 quantisation fails in
ai_edge_quantizer0.8.0 on one layer (CrossAttnDownBlock2D_2/ResnetBlock2D_0/conv2: "input operand has more dimensions than allowed by the axis remapping"). - Measured on a Pixel 10 Pro Fold (Tensor G5), CPU: DiT stages ≈ 1.1 / 3.9 / 17.3 s per call.
Credit
All credit for the model and research to Qualcomm AI Research — Animesh Karnewar and the Neodragon authors. These conversions exist only to make that work runnable on more devices.