Model Description
This is an EAGLE-3 draft model for Llama-3.1-8B-Instruct, trained from scratch using LK losses — training objectives that directly target acceptance rate rather than using KL divergence as a proxy.
Training Details
- Base model: meta-llama/Llama-3.1-8B-Instruct
- Draft architecture: EAGLE-3
- Training data: Infinity-Instruct-0625 with Llama-3.1-8B generated responses
- Training objective: Hybrid LK loss with adaptive λ scheduling (η=3)
- Training: 10 epochs from random initialization
- Draft length: K = 6 speculative tokens
Performance
Average acceptance length (τ) measured across MT-bench, HumanEval, and GSM8K with K = 7:
| Configuration | Temperature = 0 | Temperature = 1 |
|---|---|---|
| EAGLE-3 + KL | 4.36 | 3.86 |
| EAGLE-3 + LK (ours) | 4.43 | 4.01 |
Comparison with Public Checkpoints
| Model | MT-bench (τ) | HumanEval (τ) | GSM8K (τ) |
|---|---|---|---|
| yuhuili/EAGLE3-LLaMA3.1-Instruct-8B | 2.97 | 3.75 | 3.21 |
| zhuyksir/EAGLE3-Llama-3.1-8B-Instruct | 3.06 | 3.96 | 3.21 |
| Ours | 3.48 | 4.52 | 4.02 |
Measured at temperature = 1 with K = 7
Note: Earlier vLLM versions sampled draft tokens greedily regardless of temperature, which underestimated acceptance rates at temperature > 0. Stochastic draft sampling was introduced in v0.18.0, and from v0.21.0 it can be enabled via
speculative_configusingrejection_sample_methodanddraft_sample_method. The acceptance metrics reported above were measured under standard rejection sampling and are reproducible with the configuration below.
Usage with vLLM
from vllm import LLM, SamplingParams
llm = LLM(
model="meta-llama/Llama-3.1-8B-Instruct",
speculative_config={
"method": "eagle3",
"model": "nebius/EAGLE3-Llama-3.1-8B-Instruct",
"num_speculative_tokens": 6,
"rejection_sample_method": "standard",
"draft_sample_method": "gumbel",
},
)
sampling_params = SamplingParams(temperature=0.7)
outputs = llm.generate(["Explain speculative decoding in simple terms."], sampling_params)
License
This model was trained using outputs from meta-llama/Llama-3.1-8B-Instruct. Use of this model is additionally subject to the Llama 3.1 Community License Agreement.
Llama 3.1 is licensed under the Llama 3.1 Community License, Copyright © Meta Platforms, Inc. All Rights Reserved.
Citation
@misc{samarin2026lklosses,
title = {LK Losses: Direct Acceptance Rate Optimization for Speculative Decoding},
author = {Alexander Samarin and Sergei Krutikov and Anton Shevtsov and Sergei Skvortsov and Filipp Fisin and Alexander Golubev},
year = {2026},
eprint = {2602.23881},
archivePrefix = {arXiv},
primaryClass = {cs.LG},
url = {https://arxiv.org/abs/2602.23881}
}