ResonateX D1
ResonateX D1 is a small GPT-2 architecture causal language model (approximately 51M parameters, 8 layers, 8 attention heads, 512 hidden size) intended for Russian-language question/answer style text generation. It is designed to run fully offline on consumer hardware (CPU, or GPU/MPS if available).
Model Details
- Architecture: GPT-2 (
gpt2model type) - Parameters: ~51,214,848
- Layers: 8
- Attention heads: 8
- Hidden size: 512
- Vocabulary size: 50,259
- Context length: 512 tokens
- Language(s): Russian (primary); the model was trained on Russian question/answer style data and produces the most coherent output when prompted in Russian.
- License: update this field with the actual license that applies to your weights and training data before publishing.
Note: fill in the training data source, training procedure, and evaluation results sections below with accurate details before uploading this model card, as these are not derivable from the model files alone.
Intended Uses
This model is intended for:
- Experimentation with small, locally-run conversational language models.
- Educational and research use around GPT-2 style architectures.
- Lightweight offline chat applications where a large hosted LLM is not available, required, or desired.
It is not intended for:
- Production use requiring factual accuracy, safety guarantees, or content moderation, none of which this model provides on its own.
- High-stakes decision-making of any kind (medical, legal, financial, etc.).
Limitations and Bias
- As a small (~51M parameter) model, ResonateX D1 has limited world knowledge and reasoning capability compared to larger language models. Responses may be short, generic, repetitive, or factually incorrect.
- The model has a fixed context window of 512 tokens; long conversations will have earlier turns truncated.
- Any biases present in the training data are likely reflected in the model's outputs. No dedicated bias evaluation has been performed.
- The model has no built-in safety filtering. Applications built on top of it should implement their own moderation if needed.
How to Use
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
model_id = "path/to/resonatex-d1" # local path or Hugging Face repo id
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)
model.eval()
prompt = "Вопрос: Привет! Кто ты?\nОтвет:"
inputs = tokenizer(prompt, return_tensors="pt")
with torch.inference_mode():
output_ids = model.generate(
**inputs,
max_new_tokens=64,
do_sample=True,
temperature=0.7,
top_p=0.92,
top_k=50,
repetition_penalty=1.12,
pad_token_id=tokenizer.pad_token_id or tokenizer.eos_token_id,
)
response = tokenizer.decode(
output_ids[0][inputs["input_ids"].shape[1]:],
skip_special_tokens=True,
)
print(response.strip())
The model expects prompts formatted with the Russian "Вопрос: ... \nОтвет:"
(Question / Answer) template shown above. Prompting in a different format is
likely to reduce output quality, since this is the format the model was
trained on.
Local Demo Application
A local Gradio-based chat interface for this model, called ResonateX D1, is available separately. It loads the model directly from disk, requires no internet connection at runtime, and exposes temperature, max output tokens, top-p, and repetition penalty as adjustable generation settings in the UI.
Requirements: Python 3.10+, gradio, torch, transformers.
Training Data
Not documented. Add details here about the dataset(s) used to train this model, including size, source, language composition, and any preprocessing or filtering steps, before publishing this model card publicly.
Training Procedure
Not documented. Add details here about the training setup (hardware, number of steps/epochs, optimizer, learning rate schedule, hyperparameters) before publishing this model card publicly.
Evaluation
Not documented. Add any perplexity, loss curves, or qualitative evaluation results here before publishing this model card publicly.
Citation
If you use this model, please cite it as:
@misc{resonatex-d1,
title = {ResonateX D1},
author = {<add author/organization name>},
year = {<add year>},
}