Llama-3.2-3B-finance-india
Built with Llama.
meta-llama/Llama-3.2-3B-Instruct fine-tuned with LoRA to answer questions about NIFTY 50 companies and the
Indian stock market, given a relevant passage as context.
What it does
Extractive question answering over financial news. You supply a question and a context passage; it returns a short, direct answer drawn from that passage.
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("Himanshu724006/Llama-3.2-3B-finance-india", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("Himanshu724006/Llama-3.2-3B-finance-india")
prompt = ("Question: How much might Tata Steel need to pay in minerals tax dues?\n"
"Context: Tata Steel might need to pay more than Rs 17,000 crore as minerals "
"tax dues to the state of Odisha.\n"
"Answer:")
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
print(tokenizer.decode(model.generate(**inputs, max_new_tokens=40)[0],
skip_special_tokens=True))
Results
Measured against the un-fine-tuned base model on 300 held-out examples. The test split shares no source articles with training.
| Metric | Base | This model |
|---|---|---|
| Exact match | 11.7 | 76.3 |
| Token F1 | 40.8 | 91.5 |
| Numeric accuracy | 63.8 | 91.9 |
Limitations — read these
- The evaluation is optimistic. Both the training data and the test answers were generated by Qwen2.5-7B-Instruct, so these scores measure agreement with that model's answer style rather than human ground truth. The base-vs-tuned delta is a fair measure of what training changed; the absolute 91.5 F1 would not survive hand-written questions.
- Roughly 8% of numeric answers are wrong. In finance that is the error that matters. Show the source passage next to any answer so a figure can be checked, not trusted.
- No live market knowledge. It answers from the context you give it. It does not know today's prices and its training data ends at the scrape date.
- Not financial advice. It is a portfolio/educational project, not a source for investment decisions.
- English, and NIFTY 50 only. Behaviour outside that scope is untested.
Training
| Method | LoRA r=32, alpha=32, dropout=0.05, all 7 projections |
| Trainable | ~48.6M params (~1.5%) |
| Base loaded in | 4-bit NF4 + double quant, fp16 compute |
| Data | 6,368 question/answer/context triplets |
| Schedule | 1 epoch, effective batch 16, lr 2e-4 cosine |
| Hardware | 1 x Tesla T4 |
Training loss was masked to the answer tokens only, so gradient signal was not spent reproducing the question and context.
Data provenance
Built from Moneycontrol news articles for NIFTY 50 companies, discovered through the sitemaps the site publishes in its own robots.txt, then turned into question/answer/context triplets by Qwen2.5-7B-Instruct and filtered for groundedness. No article text is redistributed - only derived Q/A pairs, used for training.
Full methodology, including the sourcing and compliance decisions, is in the project repo.
License
Llama 3.2 Community License. Derivative model names must begin with "Llama", and "Built with Llama" must be displayed - both are honoured here.