Quantizations of https://huggingface.co/nvidia/OpenCodeReasoning-Nemotron-1.1-7B

Open source inference clients/UIs

• llama.cpp
• KoboldCPP
• text-generation-webui
• jan

Closed source inference clients/UIs

• LM Studio
• More will be added...

From original readme

base_model:

• Qwen/Qwen2.5-7B-Instruct datasets:
• nvidia/OpenCodeReasoning language:
• en library_name: transformers tags:
• nvidia
• code pipeline_tag: text-generation

OpenCodeReasoning-Nemotron-1.1-7B Overview

Description:

OpenCodeReasoning-Nemotron-1.1-7B is a large language model (LLM) which is a derivative of Qwen2.5-7B-Instruct (AKA the reference model). It is a reasoning model that is post-trained for reasoning for code generation. The model supports a context length of 64k tokens.

This model is ready for commercial/non-commercial use.

Results

Below results are the average of 64 evaluations on LiveCodeBench (v5) [2408-2501].

How to use the models?

To run inference on coding problems:

import transformers
import torch

model_id = "nvidia/OpenCodeReasoning-Nemotron-7B-v1.1"

pipeline = transformers.pipeline(
    "text-generation",
    model=model_id,
    model_kwargs={"torch_dtype": torch.bfloat16},
    device_map="auto",
)

prompt = """You are a helpful and harmless assistant. You should think step-by-step before responding to the instruction below.

Please use python programming language only.

You must use ```python for just the final solution code block with the following format:
```python
# Your code here
```

{user}
"""

messages = [
    {
        "role": "user",
        "content": prompt.format(user="Write a program to calculate the sum of the first $N$ fibonacci numbers")
    },
]

outputs = pipeline(
    messages,
    max_new_tokens=49152,
)
print(outputs[0]["generated_text"][-1]['content'])