The Model is ONLY PRE-TRAINED ATM
Cinimod DevOps 300M
A 287M-parameter decoder-only causal language model (Llama-3 style architecture), trained from scratch on a DevOps/ops SysAdmin domain corpus. Target usage: devops tooling assistance, ops documentation, and small on-box language modeling.
Model Details
| Property | Value |
|---|---|
| Parameters | 287,310,848 (~287M) |
| Architecture | Llama-style decoder-only (custom, not stock transformers LlamaForCausalLM params) |
| Hidden size | 1024 |
| Layers | 20 |
| Attention heads | 16 |
| KV heads (GQA) | 4 |
| Intermediate size | 2730 |
| Vocab size | 65,536 (BPE) |
| Position embeddings | RoPE, theta = 500000 |
| Trained context | 4096 tokens |
| Max context (served) | up to 256K via linear RoPE scaling |
| Embeddings | tied (no separate lm_head) |
Training
- Objective: from-scratch pretraining on a DevOps/ops corpus.
- Compute: 2x RTX 4090 (24 GB each, bf16), DeepSpeed ZeRO-2, FP32 master weights via bf16 autocast.
- Tokens: one epoch over ~132,068 sequences at seq_len 4096 (~540M tokens).
- Steps: 4000, warmup 40, LR 6e-4 cosine decay (final step LR ~0).
- Efficient attention:
torch.nn.functional.scaled_dot_product_attention(flash path via flash-attn 2). - Loss trajectory: train loss 0.43 (step 2000) -> 0.35 (step 4000).
Evaluation
- Full validation (17,492 bins / 123,656 sequences @ 4096): mean eval loss 2.3163 (perplexity 10.14). Final log in
full_val_eval.log.
Files
| File | Description | Size |
|---|---|---|
model.safetensors | Full bf16 PyTorch weights (HF format with config.json, tokenizer.json/tokenizer_config.json) | 548 MiB |
config.json | Model config (transformers) | - |
tokenizer.json / tokenizer_config.json | BPE tokenizer (vocab 65,536) | - |
train_log.log | Full training log (steps, losses, LR) | - |
full_val_eval.log | Held-out full validation eval log | - |
GGUF files are listed in the GGUF section above.
GGUF (llama.cpp) — recommended
Ready-to-serve GGUF quantizations. Both are standalone single files with no dependencies (no Cinimod source code needed). The token embedding tensor is left in BF16/F16 (the Q8_0 quantizer keeps non-32-divisible dims at F16); all other weights are as noted.
| Quantization | File | Size | Notes |
|---|---|---|---|
| Q8_0 | checkpoint-4000-Q8_0.gguf | 344 MiB | Recommended default. ~8-bit, near-lossless, ~2x smaller than F16 |
| F16 | checkpoint-4000-f16.gguf | 550 MiB | Best fidelity for llama.cpp |
How to run
HuggingFace transformers (PyTorch)
The model.safetensors require the Cinimod architecture classes (cinimod.model.llama.LlamaForCausalLM) — a custom Llama variant, not the stock transformers.LlamaForCausalLM. Load from the repo source only:
import sys
sys.path.insert(0, "/path/to/cinimod-llm/src") # package src/cinimod
from cinimod.model.llama import LlamaForCausalLM
from transformers import PreTrainedTokenizerFast
model = LlamaForCausalLM.from_pretrained("dkudos/cinimod-devops")
tok = PreTrainedTokenizerFast.from_pretrained("dkudos/cinimod-devops")
ids = tok.encode("how do I check nginx status", return_tensors="pt")
out = model.generate(ids, max_new_tokens=64)
print(tok.decode(out[0]))
If you are not in the Cinimod repo, use the GGUFs instead — they are standalone and need no source code. We publish GGUFs precisely because the HF-PyTorch path depends on the custom architecture classes.
llama.cpp (recommended for serving)
Both GGUFs load directly in llama.cpp / llama-server with no external deps.
# Q8_0 (default)
wget https://huggingface.co/dkudos/cinimod-devops/resolve/main/checkpoint-4000-Q8_0.gguf
llama-server -m checkpoint-4000-Q8_0.gguf --port 8080
# or F16 for best fidelity
wget https://huggingface.co/dkudos/cinimod-devops/resolve/main/checkpoint-4000-f16.gguf
llama-server -m checkpoint-4000-f16.gguf --port 8080
256K context via linear RoPE scaling (trained at 4096):
llama-server -m dkudos/cinimod-devops/checkpoint-4000-Q8_0.gguf \
--ctx-size 262144 --rope-scaling linear --rope-scale 64 --port 8080
Rope scaling is serve-time only; this model ships with rope_scaling: null. For aggressive 64x scaling, Yarn (--rope-scaling yarn --rope-scale 64) often generalizes better than linear if long-range coherence suffers.
One-line test:
llama-server -m checkpoint-4000-Q8_0.gguf --ctx-size 262144 --rope-scaling linear --rope-scale 64
curl http://localhost:8080/v1/chat/completions -H 'Content-Type: application/json' \
-d '{"messages":[{"role":"user","content":"List 5 common systemd service commands"}],"max_tokens":128}'
Notes on the tokenizer
Vocabulary is a 65,536-token BPE (custom, tokenizers backend). <pad>, <s>, </s>, <unk> are at indices 0-3, trained with pad_token_id=0. It is a plain causal LM — no chat template is baked in. If GGUF chat-format warnings appear they are just llama.cpp server defaults, not part of the model.
Limitations
- Pretrained from scratch on a single domain (DevOps) for one epoch at small scale (~287M) — expect domain-limited fluency, not general world knowledge.
- Exact transformers architecture classes are Cinimod-custom; use the GGUFs for maximum portability (no source code needed).
Related
dkudos/cinimod-devops-sft-base- the same trained weights in SFT-ready form:vocab_size65539 (65536 trained + 3 reserved embedding rows for chat special tokens), 287,313,920 parameters. Use that repo for fine-tuning and for the CMAF Phase-0 experiments; use the GGUFs in this repo for llama.cpp serving.
License
Apache 2.0