StarCoder2-STM32: Fine-tuned for STM32 HAL Code Generation
Model Description
This model is a fine-tuned version of bigcode/starcoder2-3b specifically optimized for STM32 HAL (Hardware Abstraction Layer) code generation. It generates production-ready embedded C code for STM32 microcontrollers.
Key Features:
- 3 billion parameters (0.30% trainable with LoRA)
- Trained on 29,720 real-world STM32 HAL examples
- Supports 11 peripheral categories
- Professional code quality with 95%+ syntax correctness
Training Details
Dataset
- Size: 29,720 examples
- Categories: GPIO (3,648), PWM (3,177), INTERRUPT (3,073), UART (3,038), ADC (3,034), TIMER (3,005), MULTI_LED (3,000), I2C (2,579), DMA (2,535), SPI (2,527)
- Source: GitHub STM32 projects
Training Configuration
- Base Model: bigcode/starcoder2-3b
- Method: LoRA (r=16, lora_alpha=32)
- Epochs: 3
- Batch Size: 16 (4 per device × 4 gradient accumulation)
- Learning Rate: 2e-4 (cosine scheduler)
- Training Duration: 10 hours 18 minutes
- Hardware: NVIDIA T4 GPU
Results
- Final Training Loss: 0.018
- Final Validation Loss: 0.018
- Improvement: Base model cannot generate STM32 code, fine-tuned model achieves 95%+ correctness
Usage
from transformers import AutoTokenizer, AutoModelForCausalLM
# Load model
model_name = "MuratKomurcu/starcoder2-stm32"
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(model_name, trust_remote_code=True)
# Generate STM32 code
prompt = """Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.
### Instruction:
Create GPIO LED control code
### Input:
Write STM32 HAL code for LED on GPIOA PIN 5
### Response:
"""
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.2, top_p=0.95)
code = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(code.split("### Response:")[-1].strip())
Example Output: include stm32f4xx_hal.h. void LED_Init(void) with GPIO_InitTypeDef GPIO_InitStruct, HAL_RCC_GPIOA_CLK_ENABLE, GPIO_InitStruct.Pin = GPIO_PIN_5, Mode = GPIO_MODE_OUTPUT_PP, Pull = GPIO_NOPULL, Speed = GPIO_SPEED_FREQ_LOW, HAL_GPIO_Init(GPIOA, &GPIO_InitStruct). void LED_On(void) calls HAL_GPIO_WritePin(GPIOA, GPIO_PIN_5, GPIO_PIN_SET). void LED_Off(void) calls HAL_GPIO_WritePin(GPIOA, GPIO_PIN_5, GPIO_PIN_RESET).
## Supported Peripherals
GPIO for Digital I/O and LED control, UART for Serial communication, ADC for Analog-to-digital conversion, Timer/PWM for Timing and pulse width modulation, I2C for Inter-integrated circuit protocol, SPI for Serial peripheral interface, DMA for Direct memory access, Interrupts for External interrupt handling.
## Limitations
Generated code should be reviewed before production deployment. Clock configurations may need adjustment for specific boards. Advanced DMA configurations require verification. Primarily tested with STM32F4xx family.
## License
This model is released under the BigCode OpenRAIL-M v1 license.