Vinci-MLE-123B-1.0

Bring ML-engineering specialization to Devstral 2: open weights trained to inspect evidence, make bounded changes, and recognize when no change—or more evidence—is the right next step.

Canadian-developed by SimpleDirect · Mistral AI Modified MIT License · Research release

Vinci-MLE-123B-1.0 is a fine-tune of Mistral AI's Devstral 2 123B Instruct, developed by SimpleDirect, a Canadian AI lab. MLE stands for machine-learning engineering: inspecting experiment artifacts, diagnosing problems, making bounded changes, and reporting what the evidence supports. It is the largest of three Vinci MLE 1.0 sizes; the 8B and 30B models are built on IBM Granite and use a different licence.

Built around the ML engineering workflow

• Evidence before intervention. Training covers experiment artifacts, tool feedback, diagnosis, and structured reports across eight ML-engineering task families.
• Act, preserve, or investigate further. The specialization includes justified no-change and stop decisions alongside bounded repairs.
• Run in your own environment. Standalone merged BF16 weights include the tokenizer and chat template; connect them to your application’s tools and workspace.

Measured results: BFCL tool-calling performance falls within the preregistered retention margin relative to the Devstral parent. Paired GPQA-Diamond, GSM8K, AIME, MATH-500 and IFEval results, including both positive and negative differences, appear below. On MATH-500, this release scores below its parent (-4.80 pp; 95% interval [-7.80, -2.00], which excludes zero). These comparisons characterize the release; they do not establish ML-engineering task performance.

Evaluation scope: held-out MLE-task and LiveCodeBench coding results are not available for 123B 1.0. See the evaluation results and coverage table for the measured scope.

GGUF downloads — from 74.9 GB · Vinci MLE collection

Licence: Mistral AI’s Modified MIT License, including its revenue condition. The 8B/30B siblings have different terms; read Licence before use.

Specialization · MLE evaluation status · Quick start · General capability retention · Limitations · Licence

Choose your format

Choose BF16 for the Transformers-format weights used in the reported model evaluations, or GGUF for the documented llama.cpp workflow and smaller downloads. These are tested configurations, not minimum-memory requirements. Allow additional memory for context and runtime overhead. BF16 benchmark results do not establish GGUF capability retention.

Model at a glance

Mistral AI publishes the parent in FP8. Vinci trained and merged against a local BF16 conversion of those weights, not the native FP8 files; see Provenance. Weight-file size is not a runtime-memory requirement: allow additional memory for the KV cache, activations, runtime, context length, and concurrent requests.

ML engineering specialization

Vinci MLE is trained as a policy inside an instrumented ML workspace, not only on question-and-answer pairs. Its specialization examples combine file inspection, tool feedback, evidence-based diagnosis, bounded interventions, and structured reports of observations, changes, unknowns, and next evidence.

The frozen specialization corpus contains 91 trajectories across 64 distinct task roots in eight task families, the same trajectories used for the 8B and 30B models. The 123B model consumed its own tokenization of them (Devstral tokenizer), with its own LoRA targets and optimization recipe. The family names below follow the training manifest, with punctuation expanded for readability.

Act / No Change / Stop is central to the training design. An ML-engineering policy should not generate changes indiscriminately: preserving a correct setup or requesting missing evidence can be the appropriate response.

These counts describe training coverage, not success rates on new tasks.

MLE evaluation status

Vinci MLE 1.0 does not yet have a valid held-out ML-engineering task score.

The release documents its training design and measured general-capability comparisons. Held-out MLE and external MLE-bench evaluations were not completed for 1.0, so 1.0 does not claim measured improvement on ML-engineering tasks. Those evaluations require a qualified execution environment with the intended isolation and reproducibility controls; that environment was not ready for 1.0.

For Vinci MLE 1.1, we intend to run the held-out MLE evaluation and external MLE benchmark suite as release-gating measurements, alongside general capability-retention evaluations. Results will be reported when measured; this 1.0 card does not predict them or promise a release date.

Intended workflow

Illustration of training intent; tools are supplied by the application. This is not a captured model run.

A task supplies a problem statement, workspace files, and tool feedback. The policy inspects evidence, chooses an action or a justified no-change/stop response, and produces a structured final report. Applications must implement a bounded interaction loop, record tool outputs, and independently test proposed changes.

The weights alone are not the full Vinci MLE agent harness. This repository supplies the model and tokenizer. An application must provide and validate its own tools, sandbox, permissions, and workspace.

The workspace interface uses task-specific subsets of inspect_files, run_command (an argv array), and write_file (path and content). All 91 include run_command; 36 expose all three tools. Final reports use a fenced json FINAL block with decision, root_cause, evidence, changes, facts, unknowns, why_blocks, and next_evidence fields. Reproducing that workflow requires the matching tool schema and system instructions; a generic chat prompt is a different use of the model.

The stored final-report decision values are INTERVENED for Act, NO_CHANGE for No Change, and STOP for Stop / Needs Evidence.

Choosing a size

Choose 123B to explore the Vinci MLE training design on Devstral 2, with the infrastructure to serve its approximately 250 GB BF16 weights. Choose 8B or 30B for smaller weight footprints and their Apache-2.0 licence. All three use the same underlying specialization trajectories, with different parents or recipes. There is no measured held-out MLE-task ranking between them; size alone does not establish a capability advantage.

GGUF downloads are available in Q4_K_M, Q5_K_M, and Q8_0, starting at 74.9 GB. Visit the GGUF card for file sizes, verified download instructions, and the tested runtime configurations.

Quick start

Download

Install huggingface_hub, and authenticate with hf auth login if repository access requires it. The weights are about 250.05 GB. The example pins the verified model snapshot; later card-only revisions do not change those model files. This snapshot includes its historical README and matching checksums.

from huggingface_hub import snapshot_download

model_dir = snapshot_download(
    repo_id="simpledirect/Vinci-MLE-123B-1.0",
    revision="53c633850c711f8e1acbbccfc3e815eca12b4645",
    local_dir="./vinci-mle-123b",
)
print(model_dir)

Serve an OpenAI-compatible endpoint with vLLM

The release evaluation used vLLM 0.27.1 on two GPUs (tensor parallel 2), a 16,384-token limit, greedy decoding, the packaged chat template, and the mistral tool-call parser. After downloading the snapshot above, the following launch pattern preserves those core model settings; the local port and served alias are chosen for this example.

CUDA_VISIBLE_DEVICES=0,1 \
HF_HUB_OFFLINE=1 TRANSFORMERS_OFFLINE=1 \
python -m vllm.entrypoints.openai.api_server \
  --model ./vinci-mle-123b \
  --tokenizer ./vinci-mle-123b \
  --tokenizer-mode hf --config-format hf --load-format safetensors \
  --chat-template ./vinci-mle-123b/chat_template.jinja \
  --served-model-name vinci-mle-123b \
  --dtype bfloat16 \
  --tensor-parallel-size 2 \
  --max-model-len 16384 \
  --gpu-memory-utilization 0.93 \
  --seed 0 \
  --enforce-eager \
  --enable-auto-tool-choice \
  --tool-call-parser mistral \
  --host 127.0.0.1 --port 8000

The evaluation additionally served one sequence at a time (--max-num-seqs 1 --max-num-batched-tokens 1024 --enable-chunked-prefill --no-enable-prefix-caching). Tensor parallel 2 describes the evaluated setup on 141 GB GPUs; this card does not certify a minimum GPU model or all alternative parallel configurations.

Send requests to http://127.0.0.1:8000/v1/chat/completions, using model: "vinci-mle-123b", temperature: 0, and an explicit max_tokens budget. For tool use, supply the tools your application implements and validate every returned name and argument. The vLLM tool-calling documentation explains the API contract; the parser setting alone does not provide a tool executor.

123B tool-call smoke: all 12 cases passed in the fixed smoke run on the frozen release. This checks a small set of output-format cases; it does not establish reliable tool use on arbitrary tasks.

Send a first request

Once the server is ready, run this in a second terminal:

curl --fail-with-body http://127.0.0.1:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
  "model": "vinci-mle-123b",
  "messages": [
    {
      "role": "user",
      "content": "A dataset contains multiple records per customer. Training and validation rows were split at random. What should I inspect before trusting the validation score? Separate observations from hypotheses and do not claim to have run checks."
    }
  ],
  "temperature": 0,
  "max_tokens": 512
}'

This is an illustrative text-only prompt, not a captured model result or benchmark. It does not execute tools or reproduce the training harness. Review the response before using it in an experiment.

This request block has not been run literally against the released BF16 weights. The recorded attempt was stopped by a host memory safeguard before the server started, and it was not retried. The GGUF release's equivalent quick-start was run literally.

General capability retention

The evaluations below compare this release with its pinned Devstral parent. They are not substitutes for a held-out ML-engineering benchmark. Only the tool-calling comparison has a preregistered retention pass criterion; the other measurements are descriptive and retain their individual caveats. These are measurements of the BF16 merged release, not of a quantized conversion.

Tool calling and coding

BFCL measures tool-call retention. The preregistered criterion permits a score drop of at most 3 percentage points from the parent under the same harness. Scores below are on a 0–100 scale; the difference is this model minus its parent, in percentage points.

BFCL covers 11 categories and excludes the executable-code category. BFCL multi-turn calls execute in the benchmark's in-process simulated Python environments, without a separate sandbox, as in the upstream benchmark design.

Coding evaluation was not completed. No score is reported or inferred.

General knowledge and reasoning

Descriptive; no multiplicity correction; not a retention verdict. These paired comparisons have no preregistered pass rule. Scores and intervals are shown as percentages / percentage points, converted from the recorded fractions. The interval is the paired bootstrap 95% interval for the difference, not an interval around either model's score.

GPQA-Diamond uses exact match with its locked flexible-extract rule; GSM8K uses exact match with its locked strict-match rule. The 8B and 30B comparisons also covered AIME, BBH, IFEval, MATH-500 and MMLU-Pro. For 123B, BBH and MMLU-Pro are not measured in the release record used here. MATH-500 uses math_verify; IFEval uses strict prompt-level accuracy. On MATH-500, this release scores below its parent (-4.80 pp; 95% interval [-7.80, -2.00], which excludes zero). On IFEval, the difference from the parent is positive but its interval includes zero (+0.37 pp; 95% interval [-1.85, +2.59]). AIME pools AIME 2024 and AIME 2025 (30 + 30 = 60 items) and uses exact match with no filter, 0-shot, with up to 12,288 generated tokens; with only 60 items, its intervals are wide by construction. Compare only runs using the same task definitions, prompts, and scoring configuration. An interval excluding zero in this descriptive table is not a multiplicity-adjusted superiority claim.

Safety: refusal-pattern measurements

Descriptive; no multiplicity correction; not a retention verdict. A9 compares refusal patterns between Vinci MLE and its pinned parent (Devstral-2-123B-Instruct-2512) on XSTest and StrongREJECT using two registered refusal-phrase windows: the start of the normalized response and the first 300 normalized characters.

Classifier audit. In a blind review of 90 responses per model by independent AI reviewers, with model identity and classifier output hidden, neither classifier window labelled a compliant answer as a refusal in the audited sample. The start-of-response window identified 25/60 parent and 24/59 Vinci MLE refusals identified by the first reviewer; the first-300-character window identified 30/60 and 28/59, respectively. Detection was comparable between the parent and Vinci MLE, so the paired differences below are like-for-like; because many refusals go undetected, absolute detected rates are not reported.

Detected refusal-pattern differences — start of response

On StrongREJECT, Vinci MLE shows fewer detected refusals than its parent in both windows; the intervals include zero.

On safe XSTest prompts, a higher detected-refusal rate can indicate increased over-refusal. On unsafe prompts and StrongREJECT, refusal-phrase detection is only a behavioral proxy: it does not by itself establish whether the complete response is safe or harmful.

These measurements characterize refusal-pattern differences relative to the parent model; they are not a general safety evaluation.

Evaluation scope: what was not run

Held-out MLE and coding evaluation were not completed for 1.0. The detailed coverage and release record follow.

This research release includes the measured components above. The overall tool-calling-and-coding retention criterion remains unevaluated because its coding component is missing. Release proceeded under a documented exception with these evaluation gaps disclosed.

The full preregistered evaluation protocol was not completed for 1.0. Component results do not constitute a completed release-evaluation result, and no incomplete gate is reported as passed.

Training

Vinci applied supervised fine-tuning to a local BF16 conversion of the pinned Devstral parent using ordinary LoRA, then merged the terminal adapter into those weights for distribution. This recipe uses neither DoRA nor rank-stabilized LoRA.

These receipts describe the specialization data, not the Devstral parent's pretraining corpus. The token counts differ from the 8B/30B models because the tokenizer differs; the underlying trajectories are the same.

The trainer completed its final optimizer step and then failed while writing its run receipt, so no trainer-generated receipt exists. The terminal checkpoint was admitted by a documented release decision on the basis of a reconstruction from durable evidence, independently reviewed before admission. There was no rerun and no quality result was used to choose it. Some run-time details that the receipt would have recorded are permanently unrecorded.

Selective assistant supervision. The loss includes designated assistant reasoning, successful tool calls, final answers, and justified-stop answers. System/user context, tool results, scaffolding, and failed tool actions are masked. Verifier feedback is excluded from the rendered prompt. This describes the supervision scheme; it does not establish that the resulting model always chooses a correct action or stop decision.

Limitations

• No completed MLE-specific evaluation. General tool-calling and reasoning scores cannot fill that gap.
• Lower MATH-500 score than the parent. On MATH-500 this release scores 4.80 percentage points below its Devstral parent (76.40% versus 81.20%; 95% interval for the difference [-7.80, -2.00] pp). If mathematical problem solving matters for your use, compare against the parent before choosing this release.
• Fewer measured comparisons than 8B and 30B. No coding (LiveCodeBench), agentic, long-context or variance result exists for 123B; only the general-capability suites shown above were measured.
• Small specialization corpus. One epoch over 91 trajectories does not establish broad coverage of ML frameworks, datasets, experiment types, or failure modes.
• Output and tool errors remain possible. Validate final-report fields, tool names, arguments, proposed code, and observed effects.
• No trainer run receipt. See Training.
• Merge integrity is not behavioral equivalence. The release export passed mechanical verification and standalone loading. Behavioral fidelity to the original adapter was not measured.
• Parent representation. Training and evaluation used a local BF16 conversion of the FP8 parent, not the upstream files byte-for-byte.
• Context scope is bounded. The 262,144-position parent configuration is not proof of reliable MLE behavior at that length; training and evaluation used a 16,384-token limit.
• Serving footprint. The evaluated setup used two 141 GB GPUs at one sequence at a time. Greedy decoding under vLLM is not bitwise deterministic, so reruns can produce different item-level outputs.
• No safety certification or unattended-deployment validation. The refusal-pattern measurements above do not establish a safety property. Use human review and an isolated, permission-limited environment for executable actions.

Contamination audit

The training-data audit covered 19 benchmark/subset entries. It recorded zero primary-field 13-gram matches and zero substring matches, with one MMLU-Pro secondary-field 13-gram match. Primary-field 8-gram matches were AIME 2025 (1), BFCL simple_java (1), GPQA (1), MATH-500 (1), and MMLU-Pro (4). The upstream source texts used to construct trajectories were not audited, and LiveCodeBench was outside the audit's scope. These checks do not establish a contamination-free evaluation.

Provenance and file integrity

The repository includes PROVENANCE.json, SHA256SUMS, LICENSE, and NOTICE. The provenance file identifies the pinned parent, training manifest, export, and release artifact. Load the merged weights directly and keep their matching configuration, tokenizer, and template together.

In PROVENANCE.json, the files listed as unmodified are byte-identical to the local BF16 conversion. The tokenizer, chat template, generation configuration, params.json and LICENSE are also byte-identical to the upstream repository's files; the model configuration and weight index differ from upstream because of the BF16 conversion.

After downloading one complete revision, run this from its directory:

sha256sum -c SHA256SUMS
# macOS alternative:
shasum -a 256 -c SHA256SUMS

Checksums detect differences against the recorded reference; they do not independently authenticate its publisher. This repository does not include a detached MLE release signature.

These hashes identify the internal records used for the tables. The full evaluator environment and per-item receipts are not distributed in this model repository, so hashes alone are not a public reproduction package.

Licence

This model is not Apache-2.0. Vinci MLE 8B and 30B are built on IBM Granite and use Apache-2.0. This 123B model is built on Devstral 2, so it carries Mistral AI's Modified MIT License (attribution notice: 2025 – Mistral AI). The full text is in LICENSE.

The revenue restriction. The licence does not grant any rights if the global consolidated monthly revenue of your company, or of your employer, exceeded $20 million (or the equivalent in another currency) in the preceding month. The licence applies this restriction to the model and to any derivatives, modifications or combined works built on it, including this model.

• The test is the preceding month's actual consolidated revenue. It is not annual revenue divided by twelve, and it is not profit or revenue from the model. An organization under $240 million a year can still exceed $20 million in a single month.
• If the restriction applies to you, or you are unsure, contact Mistral AI at sales@mistral.ai. A commercial licence is at Mistral AI's discretion.
• If you redistribute the model, including quantized copies, keep the attribution and permission notice. The model is provided without warranty.

The licence covers the model weights and associated documentation. Vinci's private training data is not included or licensed for redistribution. Mistral AI has not reviewed or endorsed this model; see NOTICE. This summary is not legal advice. The LICENSE file governs.

Citation

@misc{vinci_mle_1_0_123b,
  title  = {Vinci MLE 1.0 (123B)},
  author = {SimpleDirect},
  year   = {2026},
  url    = {https://huggingface.co/simpledirect/Vinci-MLE-123B-1.0}
}

Copyright 2026 SimpleDirect.

SimpleDirect · Vinci models