multi-qa-MiniLM-L6-cos-v1
This repository hosts the multi-qa-MiniLM-L6-cos-v1 models exported for the
React Native ExecuTorch
library as ExecuTorch .pte programs, ready to run on device.
Upstream model: multi-qa-MiniLM-L6-cos-v1
Variants
| Path | Backend | Precision |
|---|---|---|
coreml/multi_qa_minilm_l6_cos_v1_coreml_fp16.pte | coreml | fp16 |
coreml/multi_qa_minilm_l6_cos_v1_coreml_int8.pte | coreml | int8 |
vulkan/multi_qa_minilm_l6_cos_v1_vulkan_fp16.pte | vulkan | fp16 |
xnnpack/multi_qa_minilm_l6_cos_v1_xnnpack_fp32.pte | xnnpack | fp32 |
xnnpack/multi_qa_minilm_l6_cos_v1_xnnpack_int8.pte | xnnpack | int8 |
XNNPACK int8
xnnpack/multi_qa_minilm_l6_cos_v1_xnnpack_int8.pte is 8da8w (int8 dynamic activations, int8 per-channel
weights) on the Linear layers, with an int8 word-embedding table. SQuAD v1.1 dev passage
retrieval (2000 questions against their 315 paragraphs), dot product:
| top-1 | MRR | |
|---|---|---|
| fp32 | 57.50 | 0.6972 |
| int8 | 57.55 | 0.6973 |
Repository structure
config.json 47 B
coreml/config.json 1.7 kB
coreml/multi_qa_minilm_l6_cos_v1_coreml_fp16.pte 43.4 MB
coreml/multi_qa_minilm_l6_cos_v1_coreml_int8.pte 21.8 MB
tokenizer.json 695 kB
tokenizer_config.json 383 B
vulkan/config.json 947 B
vulkan/multi_qa_minilm_l6_cos_v1_vulkan_fp16.pte 43.1 MB
xnnpack/config.json 1.6 kB
xnnpack/multi_qa_minilm_l6_cos_v1_xnnpack_fp32.pte 86.2 MB
xnnpack/multi_qa_minilm_l6_cos_v1_xnnpack_int8.pte 21.9 MB
Compatibility
These files are published for the ExecuTorch v1.4.1 runtime. ExecuTorch gives no forward compatibility guarantee, so an older runtime may fail to load them.
To use them in React Native ExecuTorch, pass the model constant shipped in the library's model registry to the corresponding task pipeline. See the documentation.
To load these files in your own ExecuTorch runtime, read the compatibility note first.