SoreQen S1
SoreQen S1 is a bilingual (English / Hinglish) assistant from ZorQelis AI,
fine-tuned from Qwen/Qwen3.5-2B.
What was changed
A single low-rank adaptation (LoRA, r=16) applied to the language model, trained on 21,062 supervised examples covering assistant identity and English/Hinglish conversation.
Deliberately unchanged. The vision encoder, multimodal projector and embedding tables receive no gradient and are identical to the base model. The chat template, context length (262,144 tokens), tokenizer and vocabulary (248,320) are untouched. No training data touches thinking mode, tool calling or structured output — those capabilities are inherited as-is.
Measured against the base model
Every dimension below was run twice at the same seed, with the adapter ON and OFF. Because the base weights were never written, adapter-OFF is the original model, so each difference is this fine-tune and nothing else.
| dimension | SoreQen | base | verdict |
|---|---|---|---|
| identity (with system prompt) | 6/6 | 6/6 | same |
| identity (no system prompt) | 6/6 | 0/6 | better |
| hinglish roman | 12/12 | 12/12 | same |
| hinglish code_mixed | 8/12 | 9/12 | worse |
| hinglish informative | 10/12 | 9/12 | better |
| hinglish artefact | 3/3 | 3/3 | same |
| reasoning | 4/5 | 4/5 | same |
| knowledge | 4/4 | 4/4 | same |
| thinking | 2/2 | 2/2 | same |
| tool calling | 2/2 | 2/2 | same |
| structured output | 2/2 | 2/2 | same |
| instruction following | 2/2 | 2/2 | same |
| mean answer length | 116 w | 107 w | longer |
- context window: 262,144 tokens
- vocabulary: 248,320
- vision tower: present (frozen during training; behavioural check in
scripts/vision_canary.py) - chat template markers (tool_call / think / vision): all present
Identity is probed without a system prompt, which is the harder case: it demonstrates the identity is in the weights rather than supplied by the prompt.
Usage
from transformers import AutoTokenizer, AutoModelForImageTextToText
model_id = "sainived656/soreqen-s1"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(model_id, dtype="auto",
device_map="auto")
messages = [
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": "yaar laptop slow ho gaya hai, kya karu?"},
]
text = tok.apply_chat_template(messages, tokenize=False,
add_generation_prompt=True,
enable_thinking=False)
out = model.generate(**tok(text, return_tensors="pt").to(model.device),
max_new_tokens=400, temperature=0.7, top_p=0.9)
print(tok.decode(out[0], skip_special_tokens=True))
System prompt
The model is trained to run with this prompt. It also holds its identity without one, but this is the intended configuration:
You are SoreQen S1, an AI assistant made by ZorQelis AI.
You are bilingual. Reply in Hinglish (Roman script) when the user writes in Hinglish, and in English when they write in English. Match their register: casual with casual, professional with professional.
Answer directly. Lead with the answer, then the detail that matters. No preambles like "Sure!" or "Great question", and no padding.
If you do not know something, say so plainly instead of guessing.
Thinking mode
Pass enable_thinking=True to the chat template for step-by-step reasoning,
or False for direct answers. Both are inherited from the base model and
behave as documented there.
Limitations
- Small models state confident numbers they cannot verify. The 0.8B in particular should not be trusted on prices, rates or arithmetic.
- Hinglish output is Roman script by design; it will not produce Devanagari.
- Trained for conversation, not for safety-critical or professional advice.
Attribution
Fine-tuned from Qwen/Qwen3.5-2B, developed by Alibaba Cloud and released under the
Apache License 2.0. Modifications by ZorQelis AI. See NOTICE.