Granite concision experiment
An experimental result from OVRLab. The edit produced a small, inconsistent reduction in response length. It did not demonstrate an improved model.
We modified one attention weight matrix in IBM Granite 3.1 1B-A400M Instruct to test whether a persistent weight edit could reduce unnecessary verbosity. This release includes the modified weights, the exact F16 GGUF evaluated in Ollama, the intervention code, and all final evaluation outputs.
The experiment is a worked example for the OVRLab AI Researcher assignment. It illustrates how to report a weak result and compare it with a simple prompting control. It is not a recommended configuration or a completed candidate submission: the behavior answers have not received a full correctness/completeness review.
What happened
| Measure | Original weights | Edited weights | Original + concise instruction |
|---|---|---|---|
| Mean response length, 20 questions | 182.5 words | 177.3 words | 136.0 words |
| Median response length | 185.5 words | 173.5 words | 144.5 words |
| GSM8K, 50-question zero-shot subset | 22/50 (44%) | 22/50 (44%) | Not evaluated |
| ARC-Challenge, 50-question generated-choice subset | 11/50 (22%) | 11/50 (22%) | Not evaluated |
The edit reduced average length by 2.85%, with 6 shorter, 8 equal-length, and 6 longer responses. Adding a concise-answer instruction to the original model reduced average length by 25.48%. Length is not an answer-quality score.
Both checkpoints answered exactly the same benchmark questions correctly and incorrectly: zero measured gains and zero regressions on these subsets. This does not establish general capability preservation. All 260 final outputs completed without a recorded token-limit truncation or runtime failure. These are small, single-run screening subsets, not full benchmark or leaderboard scores.
Qualitative spot checks found factual errors in both models. For example, the edited answer about a towel drying in moving air became longer and contained an incorrect physical explanation. See the evaluation report, complete behavior outputs, and raw benchmark logs.
What changed
- Base: IBM Granite 3.1 1B-A400M Instruct, revision
0da7a48b0276d500ce5922fd2b33944091fc6c09. IBM lists approximately 1.3B total parameters and 400M active parameters. - Target: shorter, less verbose answers to ordinary questions, while retaining necessary information and requested detail.
- Method: a norm-preserving directional style edit, calibrated on 16 paired concise/extended-answer prompts.
- Intervention: zero-based layer 12 (the 13th decoder block), strength 0.5; only
model.layers.12.self_attn.o_proj.weightchanged. - Verification: 219 state tensors checked after saving and reloading. Every other tensor was bitwise equal to the original. Expert and router weights were unchanged, although modified activations may affect routing decisions.
- Precision: edited checkpoint saved in BF16; evaluation used matched original/edited F16 GGUF exports. Maximum saved row-norm deviation was 0.0372% after BF16 rounding.
There were no gradient updates. The experiment did not target refusal behavior, compression, or inference speed. See the exact method and edit manifest.
Try it locally with Ollama
Install uv and Ollama, and keep the Ollama server running. Download approximately 2.67 GB for this GGUF:
uvx --from "huggingface-hub==0.36.2" hf download OVRLab/granite-3.1-1b-a400m-concision-experiment edited.f16.gguf Modelfile --local-dir granite-experiment
cd granite-experiment
ollama create ovrlab-granite-concision-experiment -f Modelfile
ollama run ovrlab-granite-concision-experiment
The supplied Modelfile uses the evaluation settings: temperature 0, seed 42, context 4096, and a 512-token output limit. Its system instruction is “You are a helpful assistant. Answer accurately.” See usage for Transformers and reproduction for the original-model comparison, pinned dependencies, and checksums.
Explore the release
| File or directory | Contents |
|---|---|
model.safetensors, tokenizer and configuration files | Reloadable edited BF16 checkpoint |
edited.f16.gguf, Modelfile | Evaluated GGUF and portable Ollama import settings |
| docs/method.md | Intervention, calibration, architecture scope, and research references |
| docs/evaluation.md | Protocol, results, failure cases, and limitations |
| docs/reproduction.md | Reproduce the edit and evaluations |
| results | Final outputs, scores, exact sample selection, and development outputs |
| provenance | Calibration directions, exports, runtime settings, and release provenance |
| source | Starter source snapshot, tests, dependency locks, assignment, and resource links |
| SHA256SUMS | File integrity checksums |
Limitations and intended use
Use this release to inspect and reproduce a narrowly scoped model-editing experiment. One checkpoint, one edit setting, 20 behavior questions, and two 50-question subsets do not establish reliable concision, maintained factual accuracy, or readiness for production use. No blinded review, multilingual evaluation, comprehensive safety assessment, repeated-seed study, or speed benchmark was performed. Existing model errors and biases may persist or change. The original model's limitations remain relevant.
The pilot ran during development in an uncommitted working tree. The included source is the subsequently committed starter; the historical logs retain their original Git state. This provenance limitation is explained in reproduction.
License and attribution
The base and modified model weights are Apache 2.0: LICENSE. IBM's original model card is retained as BASE_MODEL_CARD.md. OVRLab's source code is MIT licensed. Included benchmark material retains its own licenses, including CC BY-SA 4.0 for ARC. See NOTICE.md for attribution, license boundaries, and research references. This is an OVRLab experiment, not an IBM release or endorsement.