QuantFactory/stable-code-instruct-3b-GGUF

This is quantized version of stabilityai/stable-code-instruct-3b created using llama.cpp

Model Description

Try it out here: https://huggingface.co/spaces/stabilityai/stable-code-instruct-3b

stable-code-instruct-3b is a 2.7B billion parameter decoder-only language model tuned from stable-code-3b. This model was trained on a mix of publicly available datasets, synthetic datasets using Direct Preference Optimization (DPO).

This instruct tune demonstrates state-of-the-art performance (compared to models of similar size) on the MultiPL-E metrics across multiple programming languages tested using BigCode's Evaluation Harness, and on the code portions of MT Bench. The model is finetuned to make it useable in tasks like,

• General purpose Code/Software Engineering like conversations.
• SQL related generation and conversation.

Please note: For commercial use, please refer to https://stability.ai/license.

Usage

Here's how you can run the model use the model:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("stabilityai/stable-code-instruct-3b", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("stabilityai/stable-code-instruct-3b", torch_dtype=torch.bfloat16, trust_remote_code=True)
model.eval()
model = model.cuda()

messages = [
    {
        "role": "system",
        "content": "You are a helpful and polite assistant",
    },
    {
        "role": "user",
        "content": "Write a simple website in HTML. When a user clicks the button, it shows a random joke from a list of 4 jokes."
    },
]

prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)

inputs = tokenizer([prompt], return_tensors="pt").to(model.device)

tokens = model.generate(
    **inputs,
    max_new_tokens=1024,
    temperature=0.5,
    top_p=0.95,
    top_k=100,
    do_sample=True,
    use_cache=True
)

output = tokenizer.batch_decode(tokens[:, inputs.input_ids.shape[-1]:], skip_special_tokens=False)[0]

Model Details

• Developed by: Stability AI
• Model type: Stable Code Instruct 3B model is an auto-regressive language model based on the transformer decoder architecture.
• Language(s): English
• Paper: Stable Code Technical Report
• Library: Alignment Handbook
• Finetuned from model: https://huggingface.co/stabilityai/stable-code-3b
• License: StabilityAI Community License.
• Commercial License: to use this model commercially, please refer to https://stability.ai/license
• Contact: For questions and comments about the model, please email lm@stability.ai

Performance

Multi-PL Benchmark:

MT-Bench Coding:

SQL Performance

How to Cite Original Model

@misc{stable-code-instruct-3b,
      url={[https://huggingface.co/stabilityai/stable-code-3b](https://huggingface.co/stabilityai/stable-code-instruct-3b)},
      title={Stable Code 3B},
      author={Phung, Duy, and Pinnaparaju, Nikhil and Adithyan, Reshinth and Zhuravinskyi, Maksym and Tow, Jonathan and Cooper, Nathan}
}