BerkeliumGPT-Coder-3B

Proudly Indian. Built for the World.

BerkeliumGPT-Coder-3B is a 3-billion-parameter coding language model developed by Berkelium AI for practical software engineering, code generation, debugging, refactoring, code completion, and developer workflows.

Built with a focus on local-first AI, the model is designed to provide capable coding intelligence while remaining practical to run on consumer and developer hardware.

Overview

The Idea

Most coding models are evaluated primarily on whether they can generate a piece of code.

BerkeliumGPT-Coder is designed around something broader:

Understand the problem. Write the code. Inspect the result. Fix it. Test it. Verify it.

This makes the model suitable not only for code generation, but also as a foundation for developer tools and coding agents.

Understand
     |
     v
  Reason
     |
     v
   Code
     |
     v
  Debug
     |
     v
 Refactor
     |
     v
   Test
     |
     v
  Verify

Capabilities

Code Generation

Generate implementations from natural-language requirements, technical specifications, and programming problems.

Code Completion

Continue functions, classes, modules, configuration files, and larger code contexts.

Debugging

Analyze source code and error messages to identify likely problems and propose fixes.

Refactoring

Improve structure, readability, maintainability, and performance while preserving intended behavior.

Code Explanation

Explain unfamiliar code, algorithms, APIs, functions, classes, and architectural patterns.

Test Generation

Create unit tests and other testing code from implementation context.

Repository Reasoning

Use surrounding source code and project structure to reason about larger software systems.

Documentation

Generate documentation, comments, docstrings, usage examples, and technical explanations.

Training

BerkeliumGPT-Coder-3B was developed through a multi-stage training pipeline focused on improving coding and software-engineering capabilities.

Base Model
    |
    v
Coding Corpus Curation
    |
    v
Filtering & Deduplication
    |
    v
Coding-Focused Retraining
    |
    v
Instruction Fine-Tuning
    |
    v
Chat / Code Formatting
    |
    v
Evaluation
    |
    v
BerkeliumGPT-Coder-3B

The training pipeline emphasizes high-quality programming data, real-world software patterns, and diverse development workflows.

The dataset pipeline is designed to prioritize:

• Real software repositories
• High-quality source code
• Multiple programming languages
• Documentation
• Tests
• Software-engineering patterns
• Code changes
• Bug fixes
• Repository context
• Deduplication
• Data provenance

Synthetic data is not used as a substitute for real software data simply to increase dataset size.

Supported Languages

The training corpus covers a broad range of programming and configuration languages.

Primary

Python
C
C++
Rust
Go
Java
JavaScript
TypeScript
C#

Additional

Kotlin
Swift
PHP
Ruby
Dart
Scala
R
Julia
Lua
Elixir
Haskell
Perl
Zig
Nim
Objective-C
Solidity
CUDA

Developer Infrastructure

SQL
Shell
PowerShell
HTML
CSS
JSON
YAML
TOML
XML
Dockerfile
Makefile
CMake
Terraform
Nix

Language coverage varies according to the underlying training data.

Quick Start

Installation

pip install -U transformers torch accelerate

Load the Model

from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "Berkelium-ai/BerkeliumGPT-Coder-3b"

tokenizer = AutoTokenizer.from_pretrained(model_id)

model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype="auto",
    device_map="auto",
)

Generate Code

prompt = """Write a Python function that performs binary search
on a sorted list and returns the index of the target."""

inputs = tokenizer(
    prompt,
    return_tensors="pt"
).to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=512,
    temperature=0.2,
    do_sample=True,
)

print(
    tokenizer.decode(
        outputs[0],
        skip_special_tokens=True
    )
)

Chat Usage

For conversational applications, use the tokenizer's native chat template when available.

messages = [
    {
        "role": "user",
        "content": "Explain this Python function and improve its error handling."
    }
]

prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)

inputs = tokenizer(
    prompt,
    return_tensors="pt"
).to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=1024,
)

response = tokenizer.decode(
    outputs[0],
    skip_special_tokens=True,
)

print(response)

Local Inference

One of the core goals of BerkeliumGPT-Coder is practical local deployment.

A 3B parameter model offers a useful balance between model capability, memory requirements, latency, and deployment cost.

The model is intended to be adaptable to environments including:

Apple Silicon
NVIDIA GPUs
AMD GPUs
CPU inference
Quantized inference
MLX
GGUF
Ollama
llama.cpp
Custom inference servers

Built for Developer Tools

BerkeliumGPT-Coder-3B is also designed as a foundation for coding applications and autonomous software-engineering systems.

A potential agent architecture looks like:

                    Berkelium Codex
                          |
                    ┌─────┴─────┐
                    |           |
                  Planner     Explorer
                    |           |
                    └─────┬─────┘
                          |
                        Coder
                          |
                        Tester
                          |
                       Reviewer
                          |
                        Verify
                          |
                    Final Result

The model can therefore serve as the reasoning and generation layer inside a larger coding runtime.

Evaluation

BerkeliumGPT-Coder should be evaluated across multiple dimensions rather than relying on a single benchmark.

Code Generation

• Functional correctness
• Syntax correctness
• Code completion
• Instruction following

Software Engineering

• Bug fixing
• Refactoring
• Test generation
• Patch generation
• Repository reasoning

General Model Behavior

• Context utilization
• Long-context reasoning
• Instruction adherence
• Hallucination rate
• Robustness

Evaluation results should always specify the model checkpoint, dataset version, prompt format, inference parameters, evaluation harness, hardware, and evaluation date.

Limitations

BerkeliumGPT-Coder-3B is a relatively compact model.

It may:

• Produce incorrect implementations
• Miss edge cases
• Hallucinate APIs
• Misinterpret requirements
• Struggle with unfamiliar libraries
• Require additional repository context
• Produce code that compiles but behaves incorrectly

Generated code should therefore be reviewed and tested before use in production environments.

For security-sensitive software, use additional static analysis, dependency scanning, testing, and human review.

Data and Provenance

The dataset pipeline is designed around traceable source material.

Where applicable, training samples retain provenance such as:

Repository
Commit
File path
Programming language
License
Source URL
Processing version

The dataset pipeline includes:

License filtering
Secret detection
Exact deduplication
Near-duplicate detection
Repository-level splitting
Contamination analysis
Quality filtering
Metadata validation

This helps make the training process reproducible and auditable.

Roadmap

Stage 1 — Foundation

• Base model integration
• Training infrastructure
• Dataset pipeline
• Model repository
• Documentation

Stage 2 — Coding Intelligence

• Coding-focused training
• Instruction formatting
• Code generation
• Code completion
• Debugging workflows
• Repository context

Stage 3 — Agentic Software Engineering

• Tool calling
• Repository navigation
• Terminal interaction
• Automated testing
• Patch generation
• Patch verification
• Multi-agent workflows
• Berkelium Codex integration

Philosophy

BerkeliumGPT is built around a straightforward principle:

Advanced AI should be capable, accessible, and practical to run.

The project focuses on building useful developer intelligence that can move beyond isolated code generation toward complete software-engineering workflows.

From a local model running on a developer workstation to an agent operating across an entire repository, the goal is the same:

Build software better.

License

BerkeliumGPT-Coder-3B is released under the Apache License 2.0.

See LICENSE for the complete license text.

Citation

If you use BerkeliumGPT-Coder-3B in your research or project, please cite the model repository.

@misc{berkeliumgptcoder3b,
  title  = {BerkeliumGPT-Coder-3B},
  author = {Berkelium AI},
  year   = {2026},
  url    = {https://huggingface.co/Berkelium-ai/BerkeliumGPT-Coder-3b}
}

Berkelium AI

BerkeliumGPT-Coder-3B

Small enough to run locally.
Built to write real software.

Proudly Indian. Built for the World.