Neutrino-Instruct

A 7B-parameter instruction-tuned language model for conversational AI

Model Card

Overview

Neutrino-Instruct is a 7B-parameter language model fine-tuned from the Neutrino base model for conversational and instruction-following tasks. It is distributed in GGUF format for efficient local inference on consumer hardware via llama.cpp, Ollama, and llama-cpp-python.

The model is designed to hold coherent, contextual dialogue across multiple turns and to follow natural-language instructions reliably at chat scale, while remaining light enough to run on a single consumer GPU or CPU-only machine.

Intended Use

Primary use cases

• Conversational assistants and chatbots
• Instruction-following agents (task completion, Q&A, summarization)
• Local/offline research prototypes where data cannot leave the device
• Educational and hobbyist LLM experimentation

Out of scope

• Medical, legal, or financial advice, or any use where model error could cause real-world harm
• Autonomous decision-making without human review
• Generation of content intended to deceive, impersonate, or manipulate
• High-stakes classification (e.g., hiring, credit, law enforcement)

Neutrino-Instruct is a general-purpose research and hobbyist model. It has not been evaluated for production or safety-critical deployment, and Fardeen NB makes no warranty as to its factual accuracy, safety, or fitness for any particular purpose.

Quickstart

llama.cpp

git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp && make

# Single prompt
./main -m ./neutrino-instruct.gguf -p "Hello, who are you?"

# Interactive chat
./main -m ./neutrino-instruct.gguf -i -p "Let's chat."

# Control output length
./main -m ./neutrino-instruct.gguf -n 256 -p "Write a poem about stars."

# Adjust temperature
./main -m ./neutrino-instruct.gguf --temp 0.7 -p "Explain quantum computing simply."

# GPU offload (if built with CUDA/Metal)
./main -m ./neutrino-instruct.gguf --gpu-layers 50 -p "Summarize this article."

Ollama

ollama run fardeen0424/neutrino

Python (llama-cpp-python)

from llama_cpp import Llama

llm = Llama(model_path="./neutrino-instruct.gguf")

response = llm("Who are you?")
print(response["choices"][0]["text"])

# Streaming
for token in llm("Tell me a story about Neutrino:", stream=True):
    print(token["choices"][0]["text"], end="", flush=True)

Training Data

Neutrino-Instruct's base model was pretrained on a mixture of web, encyclopedic, code, and long-document text, and instruction-tuned on conversational data. Component sources include:

Full dataset links are available in the metadata panel on the right side of this page.

Training procedure

Architecture

Neutrino-Instruct is built on a standard decoder-only Transformer stack. The core computations are as follows.

Scaled dot-product self-attention, applied per head:

$$\text{Attention}(Q, K, V) = \text{softmax}!\left(\frac{QK^\top}{\sqrt{d_k}}\right)V$$

with multi-head attention combining $h$ heads via a learned output projection:

$$\text{MultiHead}(Q,K,V) = \text{Concat}(\text{head}_1, \dots, \text{head}_h),W^O, \qquad \text{head}_i = \text{Attention}(QW_i^Q, KW_i^K, VW_i^V)$$

Neutrino-Instruct uses grouped-query attention (GQA): the 32 query heads are partitioned into $g=8$ groups, each group sharing a single key/value projection ($KW^K_g, VW^V_g$) instead of every query head having its own — trading a small amount of expressivity for a much smaller KV cache at inference time.

Position encoding — rotary position embeddings (RoPE), applied to query/key vectors by rotating consecutive coordinate pairs as a function of token position $m$ and frequency $\theta_i$:

$$f(x_m, m) = \big[x_m^{(1)}\cos m\theta_i - x_m^{(2)}\sin m\theta_i,\ \ x_m^{(1)}\sin m\theta_i + x_m^{(2)}\cos m\theta_i\big]$$

Feed-forward block (SwiGLU-style gating):

$$\text{FFN}(x) = \big(\text{Swish}(xW_1)\otimes xW_3\big)W_2$$

Pre-normalization (RMSNorm) around each sub-block:

$$\text{RMSNorm}(x) = \frac{x}{\sqrt{\frac{1}{n}\sum_{i=1}^n x_i^2 + \epsilon}} \odot g$$

Training objective — standard next-token cross-entropy over the vocabulary:

$$\mathcal{L} = -\sum_{t=1}^{T} \log P_\theta(x_t \mid x_{<t})$$

Quantization & System Requirements

Limitations & Risks

• Hallucination: Like all LLMs, Neutrino-Instruct can generate plausible-sounding but incorrect or fabricated information. Outputs should be verified before use in any consequential context.
• Bias: Training data drawn from web and code sources may encode social, cultural, or representational biases present in that data. The model has not undergone a formal bias audit.
• Safety tuning: No dedicated safety alignment (red-teaming, RLHF/DPO refusal training, etc.) has been performed beyond the base instruction tuning. The model may comply with harmful, unsafe, or policy-violating requests.
• Language coverage: English only; behavior on other languages is unevaluated and likely degraded.
• Context length: Supports up to 32,768 tokens; quality may degrade toward the upper end of that window, as is typical for RoPE-based models — this has not been separately verified for Neutrino-Instruct.
• No factual grounding: The model has no live access to external tools, retrieval, or the internet; all knowledge is frozen at training time.

This model has not been evaluated by a third party or independent safety team. Users deploying it in any user-facing product are responsible for their own safety evaluation, content filtering, and monitoring.

License

Released under the Apache License 2.0. See LICENSE for full terms.

Citation

@misc{fardeennb2025neutrino,
  title        = {Neutrino-Instruct: A 7B Instruction-Tuned Conversational Model},
  author       = {Fardeen NB},
  year         = {2025},
  howpublished = {Hugging Face},
  url          = {https://huggingface.co/neuralcrew/neutrino-instruct}
}

Acknowledgements

Built on top of the Neutrino base model. Thanks to the maintainers of the open datasets listed above, and to the llama.cpp, Ollama, and llama-cpp-python projects for inference tooling.