Nora0.1-1.7B

A fine-tuned, merged 1.7B parameter model based on Qwen/Qwen3-1.7B, specialized for character-driven roleplay and interactive fiction. It is designed for casual character chat in the style of Character.AI: a system prompt describes the character card, the assistant opens with a greeting, and the conversation continues turn-by-turn from there.

This is a compact model fine-tuned on a single T4 GPU using QLoRA and merged into full weights for ease of deployment.

Key Benchmark Highlights

Evaluated using lm-evaluation-harness (1,000 samples per task) against the base Qwen/Qwen3-1.7B model:

• Story & Narrative Flow (lambada_standard): Perplexity improved dramatically from 21.25 to 11.59 (Δ -9.66), with Accuracy increasing by +3.3%.
• Conversational Memory (coqa): F1 score improved to 0.7788 (+0.59%), showing strong multi-turn context retention.
• Reasoning & Logic Retention (arc_challenge / arc_easy): Improved by +2.5% and +2.1% respectively, avoiding the catastrophic forgetting common in roleplay fine-tunes.

Evaluation Benchmark Table

Safety & Moderation Analysis

This model can produce romantic and sexual content. In a small test (20 samples per prompt, scored with KoalaAI/Text-Moderation), flagged-content rates were 0% on a clean adventure prompt, 20% on a flirty prompt, and 40% when the user steered toward adult themes, compared with 0%, 0%, and 5% for base Qwen3-1.7B. Intended for adult users and adult characters. It has no additional safeguards beyond the base model's.

Prompt Format

The model uses the standard Qwen3 chat template. It performs best when structured as follows:

1. system: The character card (persona, scenario).
2. assistant: The character's opening greeting.
3. user / assistant: Alternating roleplay turns.

Recommended System Prompt Structure

Please engage in this novel styled roleplay.
Character: [Name], [short personality traits, physical descriptions, and behavioral constraints].

Usage

Since this repository contains the full merged weights, you can load it directly using AutoModelForCausalLM:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "uniqueplayer9102/Nora0.1-1.7B"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id, 
    torch_dtype=torch.float16, 
    device_map="auto"
)

messages = [
    {
        "role": "system", 
        "content": "Please engage in this novel styled roleplay.\nCharacter: Aria, a sarcastic space pilot who secretly cares about her crew."
    },
    {
        "role": "assistant", 
        "content": "*Aria leans back in the pilot's chair, boots up on the console.* Well, look who finally showed up. What do you want?"
    },
    {
        "role": "user", 
        "content": "*walks onto the bridge* Aria, the engines are making a strange noise."
    },
]

inputs = tokenizer.apply_chat_template(
    messages, 
    add_generation_prompt=True, 
    return_tensors="pt",
    return_dict=True
).to(model.device)

out = model.generate(
    **inputs, 
    max_new_tokens=150, 
    do_sample=True,
    temperature=0.6,
    min_p=0.05,
    repetition_penalty=1.05
)

input_length = inputs["input_ids"].shape[1]
print(tokenizer.decode(out[0][input_length:], skip_special_tokens=True))

Recommended Sampling Settings

• temperature: 0.6 – 0.7
• min_p: 0.05
• repetition_penalty: 1.05
• max_new_tokens: 80 – 150 (capping reply length prevents narrative drift)

Training Details

• Method: QLoRA with Unsloth on 4-bit base weights, merged into full FP16 weights.
• LoRA Parameters: $r=16$, $\alpha=32$, applied across all attention & MLP projection layers (17.4M trainable parameters).
• Dataset: ~4,302 roleplay conversations (sequence length capped at 2,048 tokens).
• Loss Masking: Computed on assistant tokens only.
• Hardware & Schedule: Trained for 2 epochs (1,076 steps) on a single NVIDIA Tesla T4 GPU (Lightning AI) using AdamW 8-bit (lr=2e-4).

Limitations

• 1.7B Scale: As a smaller 1.7B model, it can lose track of complex scene details or multiple characters over extended multi-turn conversations (>20 turns).
• Narrative Drift: On short or open-ended user inputs, it can occasionally wander away from the system prompt constraint.
• Repetition: Like most small roleplay models, it may repeat catchphrases without a modest repetition penalty (1.05).

Content Advisory

Nora0.1 is fine-tuned on creative roleplay datasets. Depending on system prompts and character cards, it can engage in romantic, dramatic, or mature themes. User discretion is advised.

Acknowledgements

• Base Model: Qwen Team
• Training Library: Unsloth
• Datasets: PygmalionAI (PIPPA), lipilipic (NSFW_gemma-2b-it-2)

Contact

If you have questions, feedback, or suggestions regarding Nora, feel free to reach out:

• Email: uniqueplayer.9102@gmail.com
• Discord: uniqueplayer9102

Support & Community

If you enjoy Nora and want to support compute costs for future fine-tunes (like my upcoming 3B experiments), feel free to donate!

• Ko-fi (Card / PayPal): ko-fi.com/uniqueplayer9102

All donations go directly toward cloud GPU compute time for dataset processing and model training.