Octen-Embedding-0.6B — DirectML ONNX

Patched ONNX export of Octen/Octen-Embedding-0.6B that runs on DirectML (Windows GPU via ONNX Runtime).

What was fixed

The original torch.onnx.export (dynamo) produces val_41 = [-1] used in Reshape shapes for multi-head attention (GQA: 16 Q heads, 8 KV heads). DirectML's execution provider cannot resolve symbolic -1 at graph-capture time.

Fix: Replace [-1] with four concrete head-count constants (16 for Q, 8 for K, 8 for V, 2048 for attention output) and reconnect 84 Reshape consumer nodes.

See fix_octen_dml.py for the full patch script.

Files

• model.fp16.onnx — ONNX graph proto (4 MB)
• model.fp16.onnx.data — external weights (1.1 GB, fp16)
• tokenizer.json — Qwen2 tokenizer
• config.json — model config (max_position_embeddings=32768)
• fix_octen_dml.py — reproduction script

Usage

import onnxruntime as ort

session = ort.InferenceSession(
    "model.fp16.onnx",
    providers=["DmlExecutionProvider"],
)

Quality

Identical to CPU fp16 reference — patch preserves quality exactly.