Qwen 3.5 2B Finance Alpaca LoRA
A LoRA fine-tuned version of Qwen 3.5 2B trained on the Finance-Alpaca dataset for finance-oriented instruction following and question answering.
This model was created as an experiment to explore parameter-efficient fine-tuning (PEFT) of a relatively small language model for financial-domain tasks.
Model Details
| Property | Details |
|---|---|
| Base Model | Qwen 3.5 2B |
| Fine-tuning Method | LoRA |
| Training Dataset | Finance-Alpaca |
| Epochs | 1 |
| Training Framework | Unsloth |
| Model Type | Causal Language Model |
| Domain | Finance |
| Language | English |
All training parameters use the default configuration provided by the training setup, with the number of training epochs explicitly set to 1.
Intended Use
This model is intended primarily for:
- Research and experimentation
- Learning and demonstrating LoRA fine-tuning
- Finance-related question answering
- Financial instruction-following experiments
- Evaluating domain adaptation of small language models
- Experimenting with parameter-efficient fine-tuning
This model is not intended to provide professional financial advice.
Training
The model was fine-tuned using Low-Rank Adaptation (LoRA).
Unlike full fine-tuning, LoRA does not update all parameters of the base model. Instead, it introduces a relatively small number of trainable parameters that learn the desired task or domain adaptation.
The training dataset used was Finance-Alpaca, which contains instruction-response examples related to financial topics.
Training Configuration
The training configuration used the default values from the training environment, with the following explicit modification:
Base model: Qwen 3.5 2B
Dataset: Finance-Alpaca
Fine-tuning method: LoRA
Epochs: 1
Other parameters: Default
Usage
Using the LoRA Adapter
If this repository contains the LoRA adapter, the original base model must be loaded first and the adapter applied on top of it.
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base_model = "Qwen/Qwen3.5-2B"
adapter_model = "siddhartha-addy-globalids-labs/qwen3.5-2b-finance-alpaca-lora"
tokenizer = AutoTokenizer.from_pretrained(base_model)
model = AutoModelForCausalLM.from_pretrained(
base_model,
device_map="auto"
)
model = PeftModel.from_pretrained(
model,
adapter_model
)
prompt = "Explain the difference between a stock and a bond."
inputs = tokenizer(
prompt,
return_tensors="pt"
).to(model.device)
outputs = model.generate(
**inputs,
max_new_tokens=256
)
response = tokenizer.decode(
outputs[0],
skip_special_tokens=True
)
print(response)
Using with PEFT
The adapter can also be loaded using the PEFT library:
from peft import AutoPeftModelForCausalLM
model = AutoPeftModelForCausalLM.from_pretrained(
"siddhartha-addy-globalids-labs/qwen3.5-2b-finance-alpaca-lora",
device_map="auto"
)
Dataset
The model was trained on the Finance-Alpaca dataset.
The dataset consists of instruction-response examples covering financial concepts and questions.
Users should review the original dataset's license and terms before using this model or creating derivatives.
Limitations
This model has several important limitations:
- It may generate incorrect financial information.
- It may produce plausible-sounding but factually incorrect answers.
- It does not have access to real-time market or financial data.
- Its knowledge is limited by the base model and fine-tuning dataset.
- One epoch of fine-tuning does not guarantee optimal performance.
- The model has not been professionally validated for financial applications.
- It should not be used as the sole source of information for investment or financial decisions.
Financial Disclaimer
This model is not a financial advisor.
Outputs generated by this model should not be interpreted as investment, trading, tax, legal, accounting, or other professional financial advice.
Always verify financial information with reliable sources and consult a qualified professional before making financial decisions.
Acknowledgements
This work builds upon the following projects and resources:
- Qwen for the base language model.
- Finance-Alpaca for the financial instruction dataset.
- Unsloth for the efficient fine-tuning workflow.
- Hugging Face for the model hosting and open-source ML ecosystem.
- PEFT for parameter-efficient fine-tuning.
License
The licensing of this model is subject to the licensing terms of the underlying Qwen 3.5 2B model and the Finance-Alpaca dataset.
Please review the original licenses before using or redistributing this model.
Citation
If you use this model in your research or experiments, please reference the original Qwen model, Finance-Alpaca dataset, and the relevant fine-tuning framework.
Organization: siddhartha-addy-globalids-labs
Model: qwen3.5-2b-finance-alpaca-lora
Fine-tuning: LoRA
Epochs: 1