gpt-oss-20b-finance
ctgt-inc/gpt-oss-20b-finance is a specialized financial reasoning model fine-tuned using self-distillation from gpt-oss-20b. It is explicitly trained to navigate complex, multi-step financial problems where real-world assumptions are ambiguous or require explicitly stated formulas. It can be served on one H100/A100 GPU.
Quickstart
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "ctgt-inc/gpt-oss-20b-finance"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.bfloat16,
device_map="auto"
)
prompt = """<|system|>
You are a financial reasoning expert. Break down the following calculation, explicitly state your underlying assumptions, and show all intermediate steps.
<|user|>
A company has a Net Income of $10M, Dividends paid of $2M, and total equity of $50M at the start of the period. Calculate the Sustainable Growth Rate (SGR) under standard assumptions.
<|assistant|>"""
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=8192, temperature=0.2)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Training & Distillation Details
This model was created via self-distillation, leveraging a synthetic dataset of multi-step reasoning traces based on financial formulas and concepts derived from Investopedia.
| Parameter | Detail |
|---|---|
| Base Model | gpt-oss-20b |
| Teacher Model | Self-distillation (gpt-oss-20b) |
| Dataset Size | 220 high-quality synthetic reasoning traces |
| Data Focus | Multi-step financial calculations with ambiguous/explicit assumptions |
| Data Source Grounding | Investopedia financial formulas & methodologies |
| Hardware | 2x NVIDIA H100 GPUs |
| Training Time | ~6 Hours |
Synthetic Dataset & Task Formulation
Rather than relying on simple question-answering, our synthetic dataset targets complex, multi-step financial reasoning. Prompts are specifically designed around edge cases and scenarios requiring explicit parameter interpretation, such as:
- Working capital adjustments
- Terminal value growth rates
- Dynamic or varying discount rates
Training Pipeline
We fine-tune using Expert LoRA paired with Hint-SD (Hint-based Self-Distillation):
- Reasoning Trace Generation: The student model generates a full step-by-step reasoning trace for a given problem.
- Teacher Localization & Hint Generation: For incorrect traces, an oracle teacher model (with access to the ground-truth answer) identifies the precise step where the error occurred and writes a targeted corrective hint.
- Localized Step Alignment: We optimize a reverse KL divergence loss between the student model output with the hint and without the hint, calculated specifically over the erroneous reasoning step.
Evaluation Results
Evaluated on the FinanceReasoning evaluation suite:
| Metric | Score |
|---|---|
| Average Accuracy | 74.79% |
| Average Execution Rate | 93.28% |
| Total Tokens Generated | 486,130 |
- Accuracy: Percentage of correct final numerical and logical reasoning outputs on test cases.
- Execution Rate: Rate at which the generated reasoning path successfully outputs an answer without exceeding the token budget.
Intended Use & Limitations
Direct Use
- Step-by-step financial formula calculations (FCF, WACC, DCF valuation models, ratios).
- Reasoning through ambiguous corporate finance problems by explicitly documenting assumptions.
- Explaining complex financial concepts and mechanics.
Limitations & Financial Disclaimer
⚠️ Notice: This model is intended for research, educational, and computational assistance purposes only. It does NOT constitute professional financial, investment, or tax advice. Always independently verify calculations and assumptions before making financial decisions.
License & Acknowledgments
This model is licensed under the Apache 2.0 License.
Special thanks and credit to the creators and open-source contributors of gpt-oss-20b, which served as the base model.