BR Sovereign LLM Local Smoke Model
Status: engineering artifact only
This is a tiny, randomly initialized Llama checkpoint trained for six CPU optimizer steps on eight original synthetic Brazilian Portuguese documents. It exists only to validate the local from-scratch training, in-process interruption/resume simulation, integrity, export, and loading paths. It does not establish recovery across a process, node, or scheduler restart.
Do not use this model for language generation, evaluation, downstream tasks, or any scientific conclusion. Its loss values are path-validation evidence, not a model-quality result. It was not trained on a supercomputer.
Measured configuration
| Field | Value |
|---|---|
| parameters | 149,696 |
| decoder layers | 2 |
| hidden size | 64 |
| attention heads | 4 |
| key/value heads | 2 |
| context length | 64 |
| observed vocabulary | 447 |
| numeric type | float32 |
| initialization seed | 1701 |
| optimizer steps | 6 |
| interruption after step | 3 |
| execution device | CPU |
The default implementation used PyTorch 2.13.0, Transformers 5.15.1, and Tokenizers 0.22.2 under Python 3.13.12.
Recovery evidence
| Measurement | Value |
|---|---|
| initial loss | 6.131303787231445 |
| uninterrupted final loss | 6.114025115966797 |
| resumed final loss | 6.114025115966797 |
| exact recovery | true |
| gradients finite and nonzero | true |
| model weights changed | true |
| Python and PyTorch RNG restored | true |
| uninterrupted state SHA-256 | c68fc0da7e6126f320bda2479d2d7de83bc3e7d09c250abe3c2ccbc1ff6b4055 |
| resumed state SHA-256 | c68fc0da7e6126f320bda2479d2d7de83bc3e7d09c250abe3c2ccbc1ff6b4055 |
| checkpoint manifest file SHA-256 | c3b64c12e3d65286586e66c6a2541b18a07729c9c8fc78038d80b560de294a88 |
| smoke report file SHA-256 | 989859964a6020b69783b71a66d761f85a8b8de5a31047b08fd3387c2a8a58f4 |
| tokenizer SHA-256 | fa8ce3f9d3e665e0caaffa380e2ee0b6d0e4f71e2084f5dda478c82e3e9f8e90 |
The full loss trajectories and final tensor hashes matched between the
uninterrupted and interrupted-then-resumed executions. The published folder
contains safetensors weights, model configuration, tokenizer, and the smoke
training report. Optimizer state and the internal .pt recovery state are not
published.
Loading
from transformers import AutoModelForCausalLM, AutoTokenizer
repo_id = "guicybercode/br-sovereign-llm-smoke"
tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForCausalLM.from_pretrained(repo_id)
Loading success does not make the generated text meaningful.
Data and license
The eight synthetic documents were drafted with AI assistance for BR Sovereign LLM, reviewed and adopted by the project owner, and released under Apache-2.0 to the extent of his rights. The smoke weights are also released under Apache-2.0. This choice does not preselect a license for future scientific weights trained on another corpus.
Contamination, memorization, and safety review
- Training input was limited to the public eight-document fixture. Its source
JSONL SHA-256 is
6896439e772f212c2848846f1f908410a606d81cd8845583024c16496a07979f; the derived Parquet SHA-256 ise2b5d4c01739e92240cfe99e6dccebce3903729d083e61e86afa37f6dd86c8a2. - No benchmark examples or third-party corpus documents are present, so there is no benchmark-contamination result to report for this engineering run.
- No formal extraction or memorization evaluation was run. With eight short training documents, reproduction of fixture phrases must be assumed possible.
- No safety evaluation was run. The model is randomly initialized and trained for only six steps; its output is not suitable for use.
- Hosted inference is disabled in the card. Loading remains documented only to verify artifact compatibility.
The scoped approval and weight-license decision are in
PUBLICATION_REVIEW.md; the machine-readable removal history is
takedown-ledger.json.
Author
Guilherme Monteiro, FIAP
ORCID: https://orcid.org/0009-0008-5294-224X