DatedGPT-2021-Instruct
DatedGPT is a family of point-in-time language models: each vintage is trained only on data available up to its cutoff date, making it suitable for lookahead-bias-free prediction and point-in-time analysis.
This is the instruction-tuned chat model with data up to 2021. For the base (pretrained) model, see datedgpt/datedgpt-2021-base.
| Property | Value |
|---|---|
| Architecture | LlamaForCausalLM |
| Parameters | ~1.3 B |
| Context length | 2048 |
| Vocab | 32,000 (SentencePiece) |
| Precision | bfloat16 |
| Data vintage | 2021 |
Chat template
The Llama-2-style chat template ships in tokenizer_config.json — apply it
with the tokenizer. The BOS token must come from the tokenizer, not as a
literal "<s>" string in your prompt text.
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
repo_id = "datedgpt/datedgpt-2021-instruct"
tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForCausalLM.from_pretrained(repo_id, torch_dtype=torch.bfloat16, device_map="auto")
prompt = tokenizer.apply_chat_template(
[{"role": "user", "content": "What is the capital of France?"}],
tokenize=False,
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=128, do_sample=True,
temperature=0.7, top_p=0.95, use_cache=True,
eos_token_id=tokenizer.eos_token_id,
pad_token_id=tokenizer.eos_token_id)
print(tokenizer.decode(output[0, inputs["input_ids"].shape[-1]:], skip_special_tokens=True))
Limitations
- Knowledge limited to the 2021 data vintage.
- No RLHF or safety tuning; outputs can be confidently wrong.