Poro 2 8B Math Reasoning SFT Preview Model Card

Poro 2 8B Math Reasoning SFT Preview is a specialized model focused on mathematical reasoning and problem-solving. This preview model was created through supervised fine-tuning of a context-extended base model. This model excels at mathematical reasoning tasks but is not optimized for general conversational use or other domains.

The model produces reasoning traces in both English and Finnish, adapting the language of its reasoning based on the language of the input prompt.

Poro 2 was created in a collaboration between AMD Silo AI, the TurkuNLP group of the University of Turku, and High Performance Language Technologies (HPLT). Training was conducted on the LUMI supercomputer, using compute resources generously provided by CSC - IT Center for Science, Finland.

For more details on our training and data generation pipeline, check out our Context Extension and Reinforcement Learning Playbook.

Poro 2 Long Model Family

The Poro 2 Long model family extends the Poro 2 models with longer context support and checkpoints trained especially on math reasoning. There are four checkpoints released: a base model, an instruction-tuned model, a math reasoning SFT checkpoint, and the final math reasoning RL checkpoint.

Model Overview

Poro 2 8B Math Reasoning SFT based on the Llama 3.1 8B architecture and has been fine-tuned for mathematical reasoning and problem-solving. The model supports both English and Finnish reasoning.

Training Process

Context Extension

The base model underwent context extension using LongRoPE scaling to support 131k tokens.

Supervised Fine-Tuning (SFT)

The SFT phase focused on mathematical reasoning and used:

• 2.2M English math reasoning samples from nvidia/Llama-Nemotron-Post-Training-Dataset
• 1.4M Finnish translations of the reasoning samples

Training Dataset

Post-Training Hyperparameters

SFT

Evaluation Results

Poro 2 8B Math Reasoning SFT Preview demonstrates strong mathematical reasoning capabilities in both English and Finnish, showing substantial improvements over the original Poro 2 8B Instruct model.

Usage

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_name = "LumiOpen/Llama-Poro-2-Long-Math-Reasoning-SFT-Preview"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

# Finnish conversation example
messages = [
    {"role": "user", "content": "Kuinka monta positiivista kokonaislukujakajaa luvulla 196 on?"}
]

inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt"
)

outputs = model.generate(
    inputs,
    max_new_tokens=32000,
    temperature=0.7,
    do_sample=True,
    pad_token_id=tokenizer.eos_token_id
)

response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

Intended Use

Poro 2 8B Math Reasoning SFT Preview is designed for:

• Mathematical problem-solving and reasoning tasks
• Step-by-step mathematical reasoning with explicit reasoning traces
• Math education and tutoring applications
• Competition mathematics (e.g., AIME-level problems)
• Mathematical reasoning in both English and Finnish

Note: This model is optimized specifically for mathematical reasoning and is not suitable for general conversational AI or other domains.

Ethical Considerations and Limitations

Poro 2 8B Math Reasoning SFT Preview is a specialized model optimized for mathematical reasoning in English and Finnish. As a preview release, this model has specific limitations:

Key limitations:

• Optimized exclusively for mathematical reasoning; not suitable for general conversation, coding, creative writing, or other domains
• This is an early preview release
• Reasoning traces are in English or Finnish depending on prompt language; limited proficiency in other languages
• May occasionally generate biased, inappropriate, or factually incorrect content

Safety Considerations:

• Users should verify important factual claims independently
• The model should not be used for medical, legal, or financial advice without human oversight
• Responses should be reviewed for appropriateness in sensitive contexts

License

Built with Llama

Poro 2 8B Math Reasoning SFT Preview is released under the Llama 3.1 Community License. Please review the license terms before use.

Citation

@misc{poro2_long_2025, title={}, author={}, year={}, howpublished={} }

Acknowledgments

We thank CSC - IT Center for Science, Finland for providing access to the LUMI supercomputer. This work was supported by the High Performance Language Technologies (HPLT) project and conducted in collaboration with TurkuNLP from the University of Turku. This project has received funding from the European Union's Horizon Europe research and innovation programme under grant agreement No 101070350.