Fleet — Llama Release

Official Fleet release. A semantic, turn-routed expert system built on Llama 3.2 3B.

Route once. Use the right specialist. Keep generation simple.

Fleet is an experimental expert-routing architecture for language models. Instead of asking one model to behave like every specialist at once, Fleet keeps a shared backbone and routes each user turn to the most appropriate expert.

This release marks the move to Llama as Fleet's main lightweight backbone and introduces the v0.12 runtime: a trained semantic router, turn-level routing, specialist adapters, expert lifecycle tooling, release validation, and utilities for converting compatible full fine-tunes into compact Fleet experts.

MAX-Fleet is also in development: a larger Qwen 9B branch aimed at pushing the architecture further. It is coming later; there are still a few side quests to finish first.

Why Fleet

A normal instruction model carries one set of behavior everywhere. Fleet separates general capability from specialized behavior.

                         ┌─ code_python
                         ├─ medical_reasoning
user turn ─► router ─────┼─ math_reasoning
                         ├─ function_calling
                         ├─ sql
                         └─ NULL / base model

The backbone is shared. Experts provide specialization. The router decides which path should handle the turn.

That gives Fleet a simple goal:

Use specialist capability only when the turn actually needs it.

Fleet v0.12 at a glance

• Base: Llama 3.2 3B Instruct
• Routing: semantic + trained + turn-based
• Experts: lightweight specialist adapters/deltas
• Default routes: Python, medical, math, function calling, SQL, and NULL/base
• Router artifact: fleet_router.safetensors
• Custom runtime: modeling_fleet.py
• Full-model conversion: compatible full fine-tunes can be approximated as LoRA-style experts
• Expert management: install, uninstall, and inspect experts from the CLI
• Validation: finite-router checks, runtime checks, and optional Triton execution checks

Fleet v0.12 is designed so the routing decision happens once per user turn, then the selected route is reused for generation.

prompt
  │
  ▼
semantic turn representation
  │
  ▼
trained router head
  │
  ├─ small domain-prior correction
  │
  ▼
selected route
  │
  ▼
generate the response

This is intentionally different from token-by-token expert routing. Routing overhead is amortized across the whole response.

Yes — Fleet uses semantics

Fleet v0.12 is not just a keyword switchboard.

The runtime builds a semantic representation of the user turn from the Llama embedding space, pools that representation into a normalized turn vector, and feeds it into a trained router head.

The simplified path is:

prompt
  → token embeddings
  → layer-0 normalization
  → pooled turn vector
  → L2 normalization
  → trained router weights + bias
  → optional domain prior
  → final expert

The learned router remains the main routing signal. Small lexical/domain priors are used as a correction layer for highly explicit cases such as SQL schemas, tool-call syntax, Python requests, medical terminology, and math prompts.

So the v0.12 router is best described as:

trained semantic turn routing with lightweight deterministic priors.

Why turn-based routing

Fleet routes once for the turn instead of repeatedly deciding during every generated token.

Turn-routed Fleet

user turn
   ↓
route once
   ↓
selected expert
   ↓
token → token → token → token → ...

The architecture is meant to keep routing predictable and cheap while still letting the model specialize by task.

This release does not claim a universal speedup percentage. Actual throughput depends on hardware, quantization, expert format, attention implementation, batch size, and whether optimized kernels are used. Benchmark on your own target runtime before making performance claims.

Reference experts

The Llama Fleet release was developed around specialists for:

The reference expert ecosystem uses models/adapters derived from the same Llama 3.2 3B family where possible.

This source repository does not redistribute third-party model weights. Each model or adapter remains subject to its own upstream license and attribution requirements.

Install

Clone the repository and install Fleet locally:

git clone YOUR_FLEET_REPO_URL
cd Fleet
python -m pip install -e .

Install optional integrations:

# Hugging Face support
python -m pip install -e '.[hub]'

# PEFT tooling
python -m pip install -e '.[peft]'

# Triton support
python -m pip install -e '.[triton]'

# Everything
python -m pip install -e '.[all]'

You can also use the included helper:

bash scripts/install.sh

To remove the local package:

bash scripts/uninstall.sh

Expert management

Fleet includes a small expert-management CLI.

List registered experts:

fleet expert list

Install/register an expert:

fleet expert install \
  --name sql \
  --source BY-ALF/llama-3.2-3b-sql-lora

Remove an expert:

fleet expert uninstall --name sql

The tooling is intentionally simple so experiments can swap specialists without rebuilding the whole project.

Using a fully fine-tuned model as a Fleet expert

Yes — Fleet can use expertise originating from a fully fine-tuned model, provided that model is architecturally compatible with the base.

For a specialist trained from the same exact base model:

ΔW = W_specialist - W_base

That dense delta can be approximated with a low-rank decomposition:

ΔW ≈ B @ A

Fleet includes a converter that performs truncated SVD on compatible two-dimensional weight matrices and exports LoRA-style low-rank factors.

fleet convert-full-to-lora \
  --base meta-llama/Llama-3.2-3B-Instruct \
  --specialist your-org/your-full-finetune \
  --output ./experts/my-specialist \
  --rank 32

Higher rank generally preserves more of the original dense fine-tune but creates a larger expert.

Conversion requirements

The base and specialist should have matching architecture, tensor names, and tensor shapes.

Good candidate:

Llama 3.2 3B base
        +
Llama 3.2 3B full fine-tune
        ↓
Fleet low-rank expert

Not directly convertible by simple weight subtraction:

Llama 3.2 3B base + Qwen 9B specialist
Llama 3.2 3B base + Mistral specialist
Llama 3.2 3B base + different-width/depth Llama

Cross-architecture specialization needs distillation, retraining, or another transfer method rather than direct delta conversion.

Load a Fleet model

A published Fleet checkpoint uses Hugging Face custom model code, so load it with trust_remote_code=True.

from transformers import AutoTokenizer, AutoModelForCausalLM

repo = "YOUR_USERNAME/Fleet-Llama-3.2-3B-v0.12"

tokenizer = AutoTokenizer.from_pretrained(
    repo,
    trust_remote_code=True,
)

model = AutoModelForCausalLM.from_pretrained(
    repo,
    trust_remote_code=True,
    device_map="auto",
)

Inspect the route

prompt = "Solve 7x - 5 = 30 and show the steps."

text = tokenizer.apply_chat_template(
    [{"role": "user", "content": prompt}],
    tokenize=False,
    add_generation_prompt=True,
)

batch = tokenizer(text, return_tensors="pt")
dev = model.get_input_embeddings().weight.device
batch = {k: v.to(dev) for k, v in batch.items()}

probs = model.fleet_route_probs(
    batch["input_ids"],
    batch.get("attention_mask"),
    prompt_text=prompt,
)[0]

route_names = [
    "NULL",
    *list(model.config.fleet_expert_names),
]

route = route_names[int(probs.argmax())]
print(route)

Release validation

Before publishing a checkpoint, validate the release directory:

fleet validate-release \
  --model-dir /path/to/Fleet-Llama-3.2-3B-v0.12

The release tooling can verify things such as:

• fleet_router.safetensors exists
• router tensors contain no NaN/Inf values
• Fleet configuration metadata is readable
• expected runtime files are present
• Triton is available when requested

The development release was also smoke-tested across the six default routes after a fresh runtime reload.

Triton

Fleet ships Triton validation tooling, but having Triton installed does not automatically accelerate the model.

Run the included GPU execution check:

python scripts/triton_check.py

A passing check proves that Triton can compile and execute a real kernel on the current GPU. Actual model speedups require Fleet's inference hot path to use optimized Triton kernels.

Treat performance as a benchmark question, not a package-presence question.

Repository layout

Fleet/
├── src/
│   └── fleet_v012/
│       ├── cli.py
│       ├── converter.py
│       ├── experts.py
│       ├── routing.py
│       └── validate.py
├── examples/
│   ├── route_once.py
│   └── convert_full_model.py
├── scripts/
│   ├── export_model_runtime.py
│   ├── install.sh
│   ├── uninstall.sh
│   └── triton_check.py
├── MODEL_CARD_TEMPLATE.md
├── RELEASE_CHECKLIST.md
├── NOTICE.md
├── pyproject.toml
└── README.md

Roadmap

Fleet — Llama

Now: the lightweight mainline release.

Focus:

• semantic turn routing
• trained router heads
• compact specialist experts
• simple expert lifecycle tooling
• full-model → low-rank conversion experiments
• predictable single-route inference

MAX-Fleet — Qwen 9B

Coming soon.

MAX-Fleet is the larger experimental branch built around a Qwen 9B-class backbone. The goal is not simply to make Fleet bigger; it is a separate place to push routing, expert quality, scaling, and runtime ideas that do not need to fit inside the lightweight Llama release.

No release date yet. There are more experiments—and a few side quests—to finish first.

What Fleet is not

Fleet is not five independent 3B models loaded side by side.

The intended design is:

shared backbone
   +
router
   +
compact expert deltas/adapters

It is also not pure keyword routing. The trained semantic router produces the learned routing signal; explicit domain priors are a small correction layer.

And Fleet is not tied forever to one backbone. The Llama release is the current lightweight mainline. MAX-Fleet explores the larger Qwen branch separately.

Status

Fleet is active experimental work. Interfaces, expert formats, routing methods, and runtime optimizations may continue to change as the project develops.

If you build an expert, improve the converter, benchmark a new runtime path, or try Fleet on another compatible checkpoint, contributions and reproducible results are welcome.

License and model weights

The source tooling in this repository is released under the license included in LICENSE.

That source license does not automatically apply to model weights. Llama weights, Qwen weights, third-party fine-tunes, and expert adapters remain governed by their respective upstream licenses and usage terms.

Before redistributing any checkpoint, verify the license and attribution requirements for every incorporated model or adapter.

Fleet

One backbone. Multiple specialists. One route per turn.

Llama Fleet is here. MAX-Fleet is next.