Personal Finance Reasoning LoRA Model

This model is a LoRA-adapted version of deepseek-ai/DeepSeek-R1-Distill-Qwen-14b, specifically fine-tuned to enhance reasoning capabilities for complex queries within the personal finance domain.

Model Details

• Developed by: Akhil-Theerthala
• Model type: Language model fine-tuned for causal language modeling and reasoning.
• Language(s) (NLP): English
• License: MIT
• Finetuned from model: deepseek-ai/DeepSeek-R1-Distill-Qwen-14b
• Model Description: Built upon the deepseek-ai/DeepSeek-R1-Distill-Qwen-14b base model, this adaptation is specifically engineered to address nuanced personal finance questions. By fine-tuning with the Akhil-Theerthala/PersonalFinance_v2 dataset, which comprises diverse scenarios requiring contextual understanding and multi-step reasoning, the model aims to generate more relevant and coherent responses to financial inquiries. The use of LoRA facilitates this specialization efficiently, targeting key aspects of financial reasoning such as query analysis and context processing inherent in the training data.
• Resources for more information:
  • Dataset: Akhil-Theerthala/PersonalFinance_v2
  • Base Model: deepseek-ai/DeepSeek-R1-Distill-Qwen-14b

Training Data

The model was fine-tuned on the Akhil-Theerthala/PersonalFinance_v2 dataset.

• Dataset Description: This dataset contains approximately ~7.04k samples designed for personal finance reasoning. It includes diverse scenarios and queries related to financial advice, planning, and problem-solving. The data is structured to facilitate tasks like question answering and text generation requiring financial reasoning, often involving analysis of context and complex query structures.
• Data Size: ~7,000 training instances.
• Format: JSON
• Source: Akhil-Theerthala/PersonalFinance_v2 on Hugging Face

Training Procedure

Adaptation Method:

The base model DeepSeek-R1-Distill-Qwen-14b (14B parameters) was adapted using Low-Rank Adaptation (LoRA). This method was chosen due to the dataset size (~7k samples), making it suitable for parameter-efficient fine-tuning rather than full fine-tuning.

Hyperparameters:

• LoRA r (rank): 64
• LoRA alpha: 128
• LoRA dropout: 0.1
• Target modules: all-linear
• Optimizer: Adam
• Learning rate: 1e-4
• Batch size: 40
• Number of epochs: 8

Evaluation

• The Evaluation is currently under process, with the major method of evaluation using LLMs-as-Judge framework, preferably Gemini-2.5-Flash as the judge. The Benchmarks used and the comparisons done will be shared shortly.

Intended Uses:

• Assisting users with personal finance queries by providing reasoned explanations and advice.
• Educational tool for understanding financial concepts.
• Generating text related to financial scenarios.
• Research in domain-specific reasoning and parameter-efficient fine-tuning.

Limitations:

• Knowledge Cutoff: The model's knowledge is limited to the information present in its training data (both the base model's pre-training and the PersonalFinance_v2 fine-tuning dataset). It may not be aware of very recent financial events, regulations, or products.
• Potential Biases: The model may reflect biases present in the training data.
• Hallucinations: Like all large language models, it may occasionally generate plausible-sounding but incorrect information (hallucinations).
• Domain Specificity: While specialized for personal finance, its reasoning capabilities might be less robust outside this domain or for highly niche financial topics not well-represented in the training data.
• Dataset Limitations: The current version of the dataset is only ~7k which is barely enough to get quality results from LoRA adaptation. Further refinement and scaling up of the dataset is necessary.

Cost Incurred: As an independent data scientist trying to develop this project, the entire cost of development till now totals up to ~$80. The cost was mostly reduced by the usage of free/open-source methods whenever available. (Though those had resource constraints with my 16GB M2 Mac Mini). Further development of this process, definitely needs some better, more cost-efficient approaches.

Further Information & Collaboration

• Contact: akhiltvsn@gmail.com
• Future Work:
  • Refining and expanding the PersonalFinance_v2 dataset. From 7k to at least 50k samples.
  • Exploring Mixture of Experts (MoE) methods for further model development.
• Call for Collaboration: I am a solo dude just randomly working on this project during my free time. If you are interested in this project, and want to expand the scope, then do ping me here, on Linkedin or just send me a mail.

Citation

@misc{akhil_theerthala_2025,
	author       = { Akhil Theerthala },
	title        = { Kuvera-14B-v0.1.0 (Revision 3fb04b9) },
	year         = 2025,
	url          = { https://huggingface.co/Akhil-Theerthala/Kuvera-14B-v0.1.0 },
	doi          = { 10.57967/hf/5707 },
	publisher    = { Hugging Face }
}