Sys1MLX
A local decision model for Apple Silicon, based on Qwen3.5-4B with a restored vision tower. Given text, an optional image, and user-supplied choices, it returns a choice and probabilities. Also supports noul and score question types.
This is a custom decision model, not a text generator. Use the included Python runtime; standard mlx_lm.generate, mlx_vlm.generate, Transformers auto-model loading, and hosted inference widgets are not supported. No new visual fine-tuning was performed. Visual decision quality is experimental.
Contents
model.safetensors: fused language weights, MLX affine Q4 (group size 64), plus the original BF16 vision encoder; 3,034,300,803 bytes (2.826 GiB).head.safetensors: FP32 pointer head; no vocabulary projection is needed.- Tokenizer, processor, config, portable Python runtime, provenance, licenses.
experimental/ane_export.py: optional fixed-geometry FP16 Core ML export recipe. Precompiled ANE models are not included.
The weights were converted from base revision 1001bb4d826a52d1f399e183466143f4da7b741b and adapter revision 70dd4088ebf4eb82d15ef57a863a5b9a98b94d6c. The base checkpoint is Qwen3.5-4B-Base, not its instruction-tuned variant. Main checkpoint SHA-256: 68efe382da6d702a44f6f9d6ebf97788eacf023eae4ef33c732b763777443580.
Run locally
Requires an Apple Silicon Mac with working MLX Metal support. Tested with Python 3.11, macOS 26.5.2, M4, 16 GiB unified memory. Smaller machines have not been validated. Allow several GiB for process memory in addition to macOS and other applications; weights alone do not determine RAM use.
Download this repository with hf download ardanila/sys1mlx --local-dir sys1mlx, then:
cd sys1mlx
python3.11 -m venv .venv
source .venv/bin/activate
python -m pip install -r requirements.txt
python -m sys1mlx.cli < example.jsonl
The supplied example has no image and needs no additional files. For visual decisions add "image_path": "/path/to/picture.png" to a request:
from sys1mlx.vision_loader import load_vision
model = load_vision(max_image_tokens=256, cache_mb=64)
result = model.predict({
"state": "Answer using the image.",
"image_path": "/path/to/picture.png",
"questions": {
"color": {
"type": "choice",
"instructions": "What color is the circle?",
"criteria": {"red": "red", "blue": "blue", "green": "green"}
}
}
})
print(result["answers"])
print(result["probabilities"]) # Unrounded probabilities, in question/option insertion order.
Keep the process alive for repeated requests. Image and optional prefix caches are bounded (64 MiB default); pass --cache-mb 0 to disable reuse or --prefix to enable context reuse. First load includes checksum verification and is excluded from steady-state latency measurements.
Limits: one still local image per request; at most 32 megapixels decoded; 16–512 image-token budget (256 default); packed image plus text at most 2,048 tokens. Aspect ratio is preserved by the Qwen processor. The head's probabilities have not been calibrated on image tasks.
Conversion recipe
- Load the pinned Qwen base and pinned LoRA adapter. Apply each adapter update in FP32 (
W + (alpha/r) * B @ A), then cast to BF16. - Quantize language linear/embedding layers to MLX affine 4-bit, group size 64. Keep the original visual tower in BF16, recurrent
A_logparameters in FP32, and convert the separate pointer head to FP32 safetensors. - Sanitize tensor names/layouts through the MLX Qwen3.5 model, retaining all visual weights. Strictly load every tensor. This artifact was converted with Linux CPU MLX 0.32.2; quantization can differ slightly from Metal conversion.
- Use the original option-token encoding and pointer readout. The runtime corrects DeltaNet Q/K normalization to the MLX-LM sum-of-squares epsilon convention; omitting this correction in mlx-vlm 0.7.2 changes outputs.
- Insert Qwen visual features and multimodal RoPE positions into the language stream. No additional training is performed.
Upstream training and benchmark results are not measurements of this quantized visual derivative. See source attribution for the original releases.
Local diagnostic results (not a general benchmark)
A small diagnostic set contained 18 synthetic color/shape/count/UI/position cases and 6 stock photographs from scikit-image. Native BF16 vision selected 24/24 expected answers, versus 7/24 without images. An image-swap control passed 18/18. These are tiny development checks, not held-out evidence of general vision accuracy; photographs may have appeared in pretraining.
Fresh-image native MLX latency was approximately 0.7–1.2 s on an M4, depending on image geometry and request. The observed MLX allocator peak was about 3.54 GiB; that is not total system memory. Repeated-image and prefix-cache timings are different workloads and must not be compared to fresh-image inference.
Experimental ANE export
python -m pip install coremltools==9.0
python experimental/ane_export.py --grid 24
python experimental/ane_export.py --grid 32
Outputs are under coreml_exports/, for processed 384×384 / 144 visual tokens and 512×512 / 256 tokens respectively. The exporter uses FP16 and fixed geometry. It exports only the visual tower; the language model and decision head still run on MLX. The default CLI continues to use BF16 MLX vision. Each package is approximately 633 MiB; leave ample free disk for Core ML compilation caches. Export/compile performance depends on macOS and hardware.
Paired local experiments using CPU_AND_NE for vision and MLX for language:
| Processed image | Native median | Hybrid median | Agreement with expected choice |
|---|---|---|---|
| 384×384 | 660 ms | 591 ms | 36/36 across 18 cases repeated twice |
| 512×512 | 1,164 ms | 1,061 ms | 4/4 across 2 cases repeated twice |
No image-token reduction was used. Maximum absolute probability change was 0.05315 (5.31 percentage points), so this is not numerically equivalent inference. The device plan preferred ANE for encoder computation; this is not hardware utilization telemetry. Initial compilation (about 26 s for one package) is excluded. Early runs encountered native compilation warnings and disk exhaustion; isolated repeated runs passed, but broad deployment reliability is unproven. Process physical footprint fell from about 3.4 to 2.8 GiB in a separate probe; this excludes memory owned by Core ML/ANE services and is not a demonstrated total-system saving.
License
Apache-2.0. See LICENSE and source attribution.