Llama 3.2 3B — Text-to-SQL
A fine-tuned version of Meta Llama 3.2 3B for converting natural-language questions into SQL queries.
The model was fine-tuned using QLoRA with a 4-bit quantized base model and LoRA adapters. After training, the adapter was merged with the base model to produce this standalone model.
Model Repositories
Merged Model
The base model and trained LoRA adapter have been merged into a standalone model:
https://huggingface.co/farehaheha/llama3.2-3B-text-to-sql
Quantized GGUF Model
A Q4_K_M GGUF quantized version is available for efficient local inference with llama.cpp and other GGUF-compatible runtimes:
https://huggingface.co/farehaheha/llama3.2-3B-text-to-sql-Q4_K_M-GGUF
Training Details
| Parameter | Value |
|---|---|
| Base Model | meta-llama/Llama-3.2-3B |
| Fine-tuning Method | QLoRA |
| Base Model Quantization During Training | 4-bit |
| LoRA Rank | 16 |
| Training Dataset | gretelai/synthetic_text_to_sql |
| Dataset Split Used | Test split |
| Training Samples | ~5.85K |
| Epochs | 1 |
| Batch Size | 8 |
| Training Time | ~30 minutes |
Note: This experiment used the dataset's test split for fine-tuning rather than the training split. Therefore, the original test split should not be used to report an unbiased evaluation score for this model.
Dataset
The model was fine-tuned on:
Gretel AI Synthetic Text-to-SQL
https://huggingface.co/datasets/gretelai/synthetic_text_to_sql
The dataset contains natural-language questions paired with SQL queries and database context.
Approximately 5,850 examples from the test split were used for this fine-tuning experiment.
Fine-Tuning Approach
The training process used QLoRA:
- The Llama 3.2 3B base model was loaded in 4-bit precision.
- The original base model weights remained frozen.
- LoRA adapters with rank 16 were trained on the Text-to-SQL dataset.
- Training was performed for one epoch.
- The trained LoRA adapter was merged with the base model.
- The merged model was also converted to Q4_K_M GGUF for local inference.
This approach significantly reduces the memory required for fine-tuning compared with full-parameter training.
Usage
from transformers import AutoTokenizer, AutoModelForCausalLM
model_id = "farehaheha/llama3.2-3B-text-to-sql"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
device_map="auto",
torch_dtype="auto",
)
prompt = """### Database Schema:
{your_database_schema}
### Request:
{your_natural_language_request}
### SQL Query:
"""
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(
**inputs,
max_new_tokens=256,
do_sample=False,
)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)