Model Card for NN-oficial
NN-oficial is an ultra-compact language model (~5.28M parameters) built upon SupraLabs/Supra-Mini-v6-1M. It incorporates an expanded vocabulary (32,000 tokens), identity-initialized middle-layer capacity expansion, and supervised instruction fine-tuning (SFT) with prompt loss masking.
Model Details
Model Description
- Developed by: jpllm
- Model type: Llama-based Causal Language Model
- Language(s) (NLP): English (
en) - License: Apache 2.0
- Finetuned from model:
SupraLabs/Supra-Mini-v6-1M
Model Sources
- Repository: https://huggingface.co/jpllm/NN-oficial
- Base Model Repository: https://huggingface.co/SupraLabs/Supra-Mini-v6-1M
Uses
Direct Use
NN-oficial is designed for research on micro-scale language models (< 10M parameters), studying parameter-efficient instruction tuning, capacity expansion, and subword vocabulary adaptation. It can generate structured, multi-line instruction completions (e.g., lists, Markdown headers, simple explanations).
Out-of-Scope Use
This model is not suitable for high-stakes decision-making, medical, legal, or factual lookup tasks due to parameter capacity constraints on world-knowledge storage.
Bias, Risks, and Limitations
Users should be aware that models under 10M parameters have physical memory bounds for storing factual world knowledge. While NN-oficial maintains strong syntactic coherence, local English grammar, and instruction response formatting, it may hallucinate factual details (e.g., exact geographic capitals or complex mathematical constants).
How to Get Started with the Model
Use the code below to load and generate text with jpllm/NN-oficial:
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
model_id = "jpllm/NN-oficial"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.float32)
prompt = (
"Below is an instruction that describes a task.\n\n"
"### Instruction:\nExplain what gravity is in simple terms for a 10-year-old.\n\n"
"### Response:\n"
)
inputs = tokenizer(prompt, return_tensors="pt")
with torch.no_grad():
outputs = model.generate(
**inputs,
max_new_tokens=100,
temperature=0.6,
top_p=0.9,
repetition_penalty=1.2,
do_sample=True,
pad_token_id=tokenizer.eos_token_id
)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
if "### Response:\n" in response:
print(response.split("### Response:\n")[-1].strip())
else:
print(response)