Model Card: TinyChat-200m-2x16

This is a fine-tuned version of TinyMoE-200m-2x16 (Mixtral architecture) optimized for chat and instruction following using LoRA.

Inference Guidelines

Important: This model does not have an embedded chat template. To ensure high-quality responses, you must manually format your prompts to match the structure used during training.

Prompt Format

The model expects the following turn-based structure:

User: [Your message here]
Assistant:

For multi-turn conversations, use:

User: [User message 1]
Assistant: [Model response 1]
User: [User message 2]
Assistant:

Python Example

from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "FlameF0X/TinyChat-200m-2x16"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)

prompt = "User: Explain how MoE works.\nAssistant:"

inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=100)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Training Details

This model was fine-tuned on the following datasets using LoRA:

1. databricks-dolly-15k: Instruction/context/response pairs.
2. no_robots: SFT dataset.
3. ultrachat_200k: SFT dataset.

Preprocessing: All datasets were converted to a standard messages format and rendered into the simple User: ... / Assistant: ... text style shown above.

Training Specs:

• Method: LoRA (Low-Rank Adaptation)
• Hardware: Trained on CPU
• Framework: Hugging Face transformers + peft
• Tokenizer: Used the base model's fast tokenizer.