llama-3.2
This repository hosts the llama-3.2 models exported for the
React Native ExecuTorch
library as ExecuTorch .pte programs, ready to run on device.
Variants
| Path | Backend | Precision | Context length |
|---|---|---|---|
1b/mlx/llama_3_2_1b_mlx_int4.pte | mlx | int4 | 16,384 |
1b/xnnpack/llama_3_2_1b_xnnpack_spinquant.pte | xnnpack | spinquant | 16,384 |
3b/mlx/llama_3_2_3b_mlx_int4.pte | mlx | int4 | 4,096 |
3b/xnnpack/llama_3_2_3b_xnnpack_spinquant.pte | xnnpack | spinquant | 4,096 |
Repository structure
1b/mlx/config.json 1.3 KiB
1b/mlx/llama_3_2_1b_mlx_int4.pte 1.1 GiB
1b/xnnpack/config.json 1.3 KiB
1b/xnnpack/llama_3_2_1b_xnnpack_spinquant.pte 1.0 GiB
3b/mlx/config.json 1.3 KiB
3b/mlx/llama_3_2_3b_mlx_int4.pte 2.3 GiB
3b/xnnpack/config.json 1.3 KiB
3b/xnnpack/llama_3_2_3b_xnnpack_spinquant.pte 2.2 GiB
config.json 31 B
tokenizer.json 9.4 MiB
tokenizer_config.json 53.2 KiB
Compatibility
These files are published for the ExecuTorch v1.4.1 runtime. ExecuTorch gives no forward compatibility guarantee, so an older runtime may fail to load them.
To use them in React Native ExecuTorch, pass the model constant shipped in the library's model registry to the corresponding task pipeline. See the documentation.
To load these files in your own ExecuTorch runtime, read the compatibility note first.
Model details
Upstream model: Llama 3.2 collection.
Both the 1B and 3B sizes are published, each as bf16 and as SpinQuant, the quantization Meta trained for on-device use.
Model details
Upstream model: Llama 3.2 collection.
Both the 1B and 3B sizes are published in three forms: XNNPACK bf16, XNNPACK SpinQuant, the quantization Meta trained for on-device use, and MLX int4.
The MLX builds quantize from the bf16 weights rather than loading a pre-quantized checkpoint, so they are independent of SpinQuant rather than derived from it.