MathGPT-2 (distilgpt2 Fine-Tuned for Arithmetic)
This model is a fine-tuned version of DistilGPT-2 on a custom dataset consisting exclusively of arithmetic problems and their answers. The goal of this model is to act as a calculator that can solve basic arithmetic problems.
Benchmark
Link here.
Model Description
The model was trained using a dataset of simple arithmetic expressions, including addition, subtraction, multiplication, and division. The training data was generated using Python and ensured to have no duplicate expressions.
Key Features:
- Solves basic arithmetic (addition, subtraction, multiplication, division)
- Can handle simple problems like
12 + 5 = - Fine-tuned version of
distilgpt2on a math-specific dataset - Trained for 10 epochs (further improvements can be made by training for more epochs)
Model Details
- Model architecture: DistilGPT-2
- Training duration: 10 epochs (could be improved further)
- Dataset: Generated math expressions like
12 + 5 = 17 - Tokenization: Standard GPT-2 tokenizer
- Fine-tuned on: Simple arithmetic operations
Intended Use
This model is designed to:
- Answer basic arithmetic problems (addition, subtraction, multiplication, division).
- It can generate answers for simple problems like
12 * 6 = ?.
Example:
Input:
13 + 47 =
Output:
60
Benchmark Results
We evaluated the model using a set of 10000 randomly generated math expressions to assess its performance. Here are the results:
- Accuracy: 76.3%
- Average Inference Time: 0.1448 seconds per question
Training Data
The training dataset was generated using Python, consisting of random arithmetic expressions (addition, subtraction, multiplication, division) between numbers from 1 to 100. The expressions were formatted as:
2 + 3 = 5
100 - 25 = 75
45 * 5 = 225
100 / 25 = 4
No duplicate expressions were used, ensuring the model learns unique patterns.
Fine-Tuning
This model was fine-tuned from the distilgpt2 base model for 100 epochs.
Limitations
- Basic Arithmetic Only: The model can only handle basic arithmetic problems like addition, subtraction, multiplication, and division. It does not handle more complex operations like exponentiation, logarithms, or advanced algebra.
- Limited Training Duration: While trained for 10 epochs, more epochs or data diversity may improve the model's performance further.
- No real-time validation: The model's performance varies, and there are still inaccuracies in answers for some problems.