Model weights from Byte-Level Elasticity: Depth through time

Details

These are the official weights of the Byte-Level Elasticity Transformer, a new kind of computation-aware architecture that serves as a baseline for researchers to discover meaningful ways to spend computation through loop-sharing the same layer weights while extrapolating to steps beyond those seen during training.

• Parameters: 10,881,809
• Source training step: 65,000
• Byte vocabulary: 259 IDs (0-255 bytes + PAD/BOS/EOS)
• Context: 512 bytes
• Physical Transformer blocks: 6
• Persistent state streams: 2
• Dynamic balanced routing
• Continuous computational phase/stride conditioning
• Default inference budget: 6 refinement cycles
• Training objective: CE(6-cycle endpoint) + 0.10 * CE(3-cycle endpoint)

For details on development of this model, you can read the official blog. This model is heavily undertrained (just ~200 million tokens) so performance might not be good enough for its size, especially for 2 and 3 cycles. Benchmarks used L6 but the author has not tested other configurations. You can find more information on how to support further research here.

Inference

For faster inference and control over loops refer to the notebook.

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "appvoid/bet-10m"

tokenizer = AutoTokenizer.from_pretrained(
    model_id,
    trust_remote_code=True,
)

model = AutoModelForCausalLM.from_pretrained(
    model_id,
    trust_remote_code=True,
    torch_dtype=torch.float16,
).cuda().eval()

prompt = "The future of artificial intelligence is"

inputs = tokenizer(
    prompt,
    return_tensors="pt",
    add_special_tokens=False,
).to(model.device)

with torch.inference_mode():
    output = model.generate(
        **inputs,
        max_new_tokens=200,
        do_sample=True,
        temperature=0.8,
        top_p=0.95,
        use_cache=True,
    )

print(tokenizer.decode(output[0], skip_special_tokens=True))

@misc{appvoid2026bet,
  title  = {Byte-Level Elasticity: Depth through time},
  author = {appvoid},
  year   = {2026},
  url    = {https://medium.com/@appvoidofficial/byte-level-elasticity-182fe2ed1d2f}
}