Vex-Amber-Mini 1.1

⚡ World Record Holder: Most Parameter-Efficient Sub-1B Language Model

Exquisitely fine-tuned from Qwen3-0.6B for unparalleled code generation and versatile text processing.

Overview

Vex-Amber-Mini 1.1 is a groundbreaking small language model (SLM) that holds the world record for the most parameter-efficient model with fewer than 1 billion parameters. Meticulously optimized for code generation and general-purpose text tasks, it delivers exceptional performance within a compact 0.6B parameter framework.

• Base Model: Qwen3-0.6B
• Fine-tuning: LoRA with optional full-weight fine-tuning for enhanced adaptability
• Parameter Count: 0.6B (preserved for maximum efficiency)
• Frameworks: Hugging Face Transformers, PyTorch
• Files: .safetensors, tokenizer.json

Installation

To harness the power of Vex-Amber-Mini 1.1, install the required dependencies:

pip install transformers torch

Ensure Python 3.8+ and the latest versions of the required libraries for seamless compatibility.

Usage Example

Experience the model’s elegance with this example of generating a Python function:

from transformers import AutoModelForCausalLM, AutoTokenizer

# Initialize the tokenizer and model
tokenizer = AutoTokenizer.from_pretrained("Arioron/Vex-Amber-Mini-1.0")
model = AutoModelForCausalLM.from_pretrained("Arioron/Vex-Amber-Mini-1.0")

# Craft the input prompt
prompt = "Write a Python function to compute Fibonacci numbers:"
inputs = tokenizer(prompt, return_tensors="pt")

# Generate refined output
outputs = model.generate(**inputs, max_length=100, temperature=0.7)

# Decode and present the result
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Model Details

• Architecture: Transformer-based, derived from Qwen3-0.6B
• Training Data: Fine-tuned on a curated dataset optimized for code generation and versatile text tasks
• Performance: Achieves a HumanEval Pass@1 score of 20.12%, setting a benchmark for sub-1B models and earning the title of the most parameter-efficient sub-1B model
• Use Cases: Ideal for code generation, text completion, and lightweight NLP applications
• Context Length: Supports up to 2048 tokens for efficient processing

Performance Metrics

The following table compares the HumanEval performance of Vex-Amber-Mini 1.1 against other code generation models. Note that scores for rival models are approximate, as indicated by "~", based on available benchmarks:

Note: The HumanEval Pass@1 score reflects the model's ability to generate correct code solutions on the first attempt. Vex-Amber-Mini 1.0 achieves competitive performance for its size, outperforming larger models in parameter efficiency.

Limitations

• Compact Design: The 0.6B parameter count, while highly efficient, may limit performance on highly complex tasks compared to larger models.
• Domain-Specific Fine-tuning: Optimal results may require additional tuning for specialized applications.
• Context Constraints: Limited to 2048 tokens, which may impact performance in extended context scenarios.

License

This project is proudly licensed under the Apache 2.0 License, ensuring open and flexible usage.

Acknowledgments

• Built upon the robust foundation of Qwen3-0.6B.
• Gratitude to the Hugging Face team for their exceptional Transformers library.
• Heartfelt thanks to the open-source community for their invaluable contributions and feedback.

Contact

For inquiries, collaboration, or to report issues, please visit:

• Hugging Face Repository: Arioron/Vex-Amber-Mini-1.0
• GitHub Repository: Vex-Amber-Mini-1.0
• Community Discussion: Join the conversation on Hugging Face Discussions

Contribute

We warmly welcome contributions! Please submit pull requests or issues via the GitHub repository to help refine and elevate Vex-Amber-Mini 1.1.