Lacaille-MoT-4B-Supreme2

Lacaille-MoT-4B-Supreme2 is a high-efficiency, multi-domain model fine-tuned on Qwen3-4B using the Mixture of Thoughts (MoT) dataset enhanced with code, math, science expert clusters and an extended open code reasoning dataset. This model blends symbolic precision, scientific logic, and structured output fluency—making it an ideal tool for developers, educators, and researchers seeking advanced reasoning under constrained compute.

[!note] GGUF: https://huggingface.co/prithivMLmods/Lacaille-MoT-4B-Supreme2-GGUF

Key Features

1. Unified Reasoning Across Code, Math & Science Fine-tuned on MoT expert clusters spanning programming, mathematics, and scientific logic, alongside an open code reasoning dataset, boosting multi-modal symbolic reasoning.

2. Advanced Code Reasoning & Generation Supports multi-language coding with explanations, optimization hints, and error detection—ideal for full-stack prototyping, algorithm synthesis, and debugging workflows.

3. Scientific Problem Solving Performs analytical reasoning in physics, biology, and chemistry—explaining concepts, solving equations, and handling symbolic derivations step-by-step.

4. Hybrid Symbolic-AI Thinking Combines structured logic, chain-of-thought reasoning, and open-ended inference, delivering robust performance on STEM tasks and complex prompt decomposition.

5. Structured Output Mastery Seamlessly generates output in LaTeX, Markdown, JSON, CSV, and YAML, suited for research reports, technical documentation, and data formats.

6. Optimized 4B Footprint for Versatile Deployment Strikes a balance between performance and efficiency, making it deployable on mid-range GPUs, offline clusters, and advanced edge AI systems.

Quickstart with Transformers

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "prithivMLmods/Lacaille-MoT-4B-Supreme2"

model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype="auto",
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(model_name)

prompt = "Explain the difference between Newtonian mechanics and quantum mechanics with examples."

messages = [
    {"role": "system", "content": "You are a scientific tutor skilled in code, math, and reasoning."},
    {"role": "user", "content": prompt}
]

text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)

model_inputs = tokenizer([text], return_tensors="pt").to(model.device)

generated_ids = model.generate(
    **model_inputs,
    max_new_tokens=512
)
generated_ids = [
    output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
]

response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
print(response)

Intended Use

• Scientific tutoring, computational logic, and mathematical education
• Advanced coding assistant for algorithm design, code reviews, and documentation
• Structured technical data generation across formats and fields
• STEM-focused chatbot or API for research and education tools
• Mid-resource deployment requiring high symbolic fidelity

Limitations

• Not tuned for general-purpose or long-form creative writing
• Context limitations may hinder multi-document or full codebase analysis
• Specialized in technical and symbolic tasks—general chat may underperform
• Prioritizes structured reasoning over emotional or casual tone generation

References

1. Qwen2.5 Technical Report (2024)
2. YaRN: Efficient Context Window Extension of Large Language Models
3. open-r1/Mixture-of-Thoughts
4. Open Code Reasoning Dataset (2024)