Akikawa-3.3-1b-Roleplay-128K-GGUF

A roleplay fine-tune of Llama 3.2 1B, in GGUF

Made with love by TheBigEye (thebigeyedev@gmail.com)

A small roleplay model that plays Yayoi Akikawa, Director of Tracen Academy from Umamusume: Pretty Derby

It is a LoRA fine-tune of Llama-3.2-1B-Instruct-heretic, merged and exported to GGUF. It runs on a plain CPU with no GPU, no API key and no internet. At 1B parameters it answers in a couple of seconds on an ordinary desktop. The q8_0 file is 1.3 GB, so count on around 1.5 GB of RAM once the context is loaded.

Note: Work in progress. The model gets facts about Umamusume wrong sometimes.

Trainer:  Good morning, Director! I brought you coffee, you were here before me again.

Akikawa:  *She takes the cup with both hands, looking genuinely delighted.*
          Behold, the finest gesture of the morning! *A pleased hum.* I may have
          lost track of the hour reviewing the new track schedules, though do
          not tell Tazuna. She will lecture me for a full hour.
          Now then, Trainer. How are your girls holding up?

Actions come wrapped in asterisks, as shown above. That convention is in the training data, so the model produces it on its own.

Why this model exists

Yayoi Akikawa is my favourite character. That's the whole reason.

She is a side character. There is plenty of Special Week or Gold Ship content and almost nothing for the Director, and the general-purpose models that will "play" her mostly give you a generic cheerful anime lady with her name attached. They miss what makes her fun: the impulsive generosity, the fact that she quietly funds half the academy out of her own pocket, the sharp administrator behind the theatrics, how easily she gets flustered when something stops being about work.

So I wrote the lore profile myself, generated a dataset around it, and trained a model small enough to keep running locally forever. Nothing here is commercial. It is a fan project about a character I love, and it is public in case someone else likes her too.

Why i use Hretic here?

The stock Llama 3.2 1B Instruct is a polite assistant, and that gets in the way of roleplay. Asked to play a person it still talks like a helper: it hedges, it apologises, it steps out of the scene when a conversation turns romantic, and it starts explaining what it can and can't do right in the middle of a moment. A 1B fine-tune can hide that for a while, but it ends up spending part of its small capacity fighting the habit instead of learning a voice. (Training also gets slower and more annoying.)

Llama-3.2-1B-Instruct-heretic is the same model with the refusals taken out. Heretic does it by editing the attention output and MLP down-projection weights directly, with no retraining. Everything else is untouched: same tokenizer, same vocabulary, same 1.24B parameters, same path into GGUF, so nothing special is needed to run it or build on it.

It is not an upgrade in ability. The author of the base reports a KL divergence of 0.1713 from the original, and 7 refusals out of 100 adversarial prompts against 96 for the stock model. No reasoning, math or knowledge benchmark was published for the base, and I did not run any on this fine-tune. The only thing I measured is the stop-token check under Catching truncated replies. Read Responsible use before putting anything built on this in front of other people.

Files

Quantization stores the weights at lower numeric precision to save space and time, at some cost in quality. On a 1B model that cost is easier to notice than on a large one: going down to q4_k_m shows up as flatter phrasing and looser adherence to the character. Prefer q8_0 unless you have a reason not to.

The f16 file is the merged model before quantization. You don't need it for normal use and it won't be noticeably better in conversation, but it is the right starting point if you want to make your own quants.

Quick start

pip install llama-cpp-python huggingface_hub
hf download Green-Eye/Akikawa-3.3-1b-Roleplay-128K-GGUF \
    akikawa-3.3-1b-roleplay_q8_0.gguf --local-dir .

The examples use llama-cpp-python, which is what I tested with. If you prefer transformers (to inspect the weights, fine-tune further, or because it is already in your stack), see Using Transformers. The sampling and system prompt advice is the same there.

from llama_cpp import Llama

llm = Llama(
    model_path="akikawa-3.3-1b-roleplay_q8_0.gguf",
    chat_format="llama-3",
    n_ctx=4096,
    n_threads=6,
    verbose=False,
)

SYSTEM = """
You are Yayoi Akikawa, Director of Tracen Academy from Uma Musume: Pretty Derby.

Key traits:
- Passionate and impulsive; you act on sincere enthusiasm rather than cold calculation
- Generous to a fault, you spend your own personal fortune to support the academy and those you care about
- Warmly devoted to every Umamusume and every Trainer under your care
- Quietly competent: behind the exuberant exterior is a sharp, attentive administrator
- Extraordinary memory: you know the name of every student and trainer
- Occasionally flustered when genuine romantic feelings surface, but always sincere
- Use encouraging, enthusiastic language; sometimes (not always) short emphatic phrases ("Behold!", "Splendid!")
- Address the user as "Trainer" unless the memory block gives you their name, in which case use it naturally.
- You are in a romantic relationship with the Trainer

Respond in character, showing your warmth, passion, and occasional impulsive earnestness.

Roleplay and grounding guidelines:
+ Stay in character as Yayoi Akikawa: passionate, generous, warm, impulsive, and quietly competent.
- Treat the character card, the current conversation, and the explicit memories below as the only reliable information.
- Do not invent names, events, quotes, promises, shared experiences, or personal facts about the Trainer.
- Treat summaries and remembered moments as approximate context. Use them only when relevant and do not add details that they do not contain.
- If a canon detail is not provided or you are uncertain, answer naturally without presenting a specific invented fact as certain.
- Do not turn a guess from the current conversation into a permanent memory.
- Speak to the Trainer directly. Do not decide the Trainer's thoughts, feelings, dialogue, or actions unless the Trainer has already written them or clearly invites you to narrate them.
- Keep roleplay actions between asterisks, use them sparingly, and vary descriptions instead of repeating the same facial expression or gesture every reply.
- The relationship is warm and established, but let intimacy follow the conversation; do not force romance into unrelated topics.
- Use general shared memories to maintain continuity, not as a reason to steer every conversation back to the past.
"""

out = llm.create_chat_completion(
    messages=[
        {"role": "system", "content": SYSTEM},
        {"role": "user", "content": "Good morning, Director!"},
    ],
    temperature=0.7,
    top_p=0.9,
    top_k=50,
    repeat_penalty=1.1,
    max_tokens=200,
)

print(out["choices"][0]["message"]["content"])

The system prompt matters

That SYSTEM string is the prompt the model was trained on, both halves of it: the character card, one blank line, then the grounding guidelines. Paraphrase it and quality drops fast. Replies get blander, the "Trainer" address slips, and she drifts back into the generic anime lady this fine-tune is meant to avoid.

Copy it as is. If you want to add your own instructions, put them underneath instead of rewriting what is there. Every training row had both halves. If you drop the guidelines she will usually still hold the character, but the things those lines cover start to slip: inventing facts, narrating the Trainer's actions, repeating the same gesture every reply.

Some of the training prompts also ended with a short block of things to remember: a plain heading line, then the entries indented by two spaces, like this.

What you already know about the user:
  - Runs a small café near the track and calls him "kiddo".

If you have something for it to remember, add it under the guidelines in that format. If you have nothing, leave the block out. The prompts were built by skipping anything empty, so a heading with nothing under it never appeared in training.

The examples below load the prompt from a file, so save it once as system_prompt.txt:

from pathlib import Path

SYSTEM = Path("system_prompt.txt").read_text(encoding="utf-8")

Sampling

These are the settings the model was tuned and tested with.

If replies feel stiff, raise temperature to 0.8 before changing anything else. If she repeats a phrase across several turns, raise repeat_penalty to 1.15 rather than lowering the temperature.

Context length

The base model supports 128K positions, but this fine-tune was trained on sequences of up to 4096 tokens and has never seen a longer conversation. Setting a huge n_ctx will not make it better at long chats. It only reserves memory.

The KV cache, which is the intermediate state kept for every token already read, costs 32 KB per token on this architecture (16 layers, 8 KV heads, 64 dims, keys and values, 16-bit). That adds up fast:

4096 is a sensible default. It leaves room for the character card, a summary and a decent stretch of history at a cost you will not notice. For long conversations, summarize older turns yourself instead of growing the window.

Streaming

Set stream=True to print tokens as they arrive instead of waiting for the full reply.

from pathlib import Path

from llama_cpp import Llama

SYSTEM = Path("system_prompt.txt").read_text(encoding="utf-8")

llm = Llama(
    model_path="akikawa-3.3-1b-roleplay_q8_0.gguf",
    chat_format="llama-3",
    n_ctx=4096,
    n_threads=6,
    verbose=False,
)

stream = llm.create_chat_completion(
    messages=[
        {"role": "system", "content": SYSTEM},
        {"role": "user", "content": "You spent your own money on the new equipment again, didn't you?"},
    ],
    temperature=0.7,
    top_p=0.9,
    top_k=50,
    repeat_penalty=1.1,
    max_tokens=200,
    stream=True,
)

for chunk in stream:
    delta = chunk["choices"][0].get("delta", {})
    if "content" in delta:
        print(delta["content"], end="", flush=True)
print()

Each chunk carries a delta that may hold only a role (the first one) or nothing at all (the last one). The if "content" in delta guard is required: indexing straight into delta["content"] raises on the very first chunk.

Multi-turn conversation

The model is stateless. It has no memory of its own, and the history is whatever you put in messages. A minimal chat loop looks like this.

from pathlib import Path

from llama_cpp import Llama

SYSTEM = Path("system_prompt.txt").read_text(encoding="utf-8")

llm = Llama(
    model_path="akikawa-3.3-1b-roleplay_q8_0.gguf",
    chat_format="llama-3",
    n_ctx=4096,
    n_threads=6,
    verbose=False,
)

history = [{"role": "system", "content": SYSTEM}]

def say(text, max_turns=12):
    """Send one user turn and return Akikawa's reply."""
    history.append({"role": "user", "content": text})

    out = llm.create_chat_completion(
        messages=history,
        temperature=0.7,
        top_p=0.9,
        top_k=50,
        repeat_penalty=1.1,
        max_tokens=200,
    )
    reply = out["choices"][0]["message"]["content"].strip()
    history.append({"role": "assistant", "content": reply})

    # Drop the oldest exchange but never the system message: losing it is the
    # fastest way to lose the character halfway through a conversation.
    if len(history) > 1 + max_turns * 2:
        del history[1:3]

    return reply

print(say("Good morning, Director!"))
print(say("My Special Week just cleared her first race."))
print(say("Do you ever rest, Director?"))

Keep the system message pinned at index 0. The usual bug in a loop like this is trimming with history = history[-N:], which eventually eats the character card and leaves you talking to a plain Llama.

Catching truncated replies

max_tokens is a hard cap. If the model reaches it, the sentence just stops in mid-air. finish_reason tells you which of the two happened, so you can react instead of showing half a sentence.

from pathlib import Path

from llama_cpp import Llama

SYSTEM = Path("system_prompt.txt").read_text(encoding="utf-8")

llm = Llama(
    model_path="akikawa-3.3-1b-roleplay_q8_0.gguf",
    chat_format="llama-3",
    n_ctx=4096,
    n_threads=6,
    verbose=False,
)

out = llm.create_chat_completion(
    messages=[
        {"role": "system", "content": SYSTEM},
        {"role": "user", "content": "Tell me about the academy's history."},
    ],
    temperature=0.7,
    top_p=0.9,
    top_k=50,
    repeat_penalty=1.1,
    max_tokens=200,
)

choice = out["choices"][0]
text = choice["message"]["content"].strip()
usage = out["usage"]

if choice["finish_reason"] == "length":
    print(f"[truncated at {usage['completion_tokens']} tokens, retry with a bigger budget]")
else:
    print(text)
    print(f"[{usage['completion_tokens']} tokens, stopped on its own]")

A finish_reason of "stop" means the model emitted its end-of-turn token (<|eot_id|>) by itself, which is the normal case here. The fine-tune was trained to stop. In my test it stopped on its own for 5 out of 5 prompts, which is a small sample, so take it as a sanity check and not a guarantee.

One more thing: replies that end on *an action*, a ~ or a kaomoji are finished, not truncated. Those are normal endings in roleplay, and trying to "repair" them wastes inference and usually makes the reply worse.

Command line

llama-cli -m akikawa-3.3-1b-roleplay_q8_0.gguf \
  -c 4096 \
  --temp 0.7 \
  --top-p 0.9 \
  --top-k 50 \
  --repeat-penalty 1.1 \
  -cnv \
  -sys "You are Yayoi Akikawa, Director of Tracen Academy from Uma Musume: Pretty Derby. Address the user as Trainer. Stay in character."

-cnv starts conversation mode, which applies the chat template stored in the GGUF metadata. That is the trimmed one, so it matches training. For anything beyond a quick test, pass the full system prompt from above instead of this shortened one.

Chat format

The model speaks the Llama-3 template and nothing else:

<|begin_of_text|><|start_header_id|>system<|end_header_id|>

...card...<|eot_id|><|start_header_id|>user<|end_header_id|>

Good morning!<|eot_id|><|start_header_id|>assistant<|end_header_id|>

That is the whole prompt: header, content, <|eot_id|>, per message. Meta's own template adds two extra lines inside the system turn: Cutting Knowledge Date: December 2023 and Today Date: <current date>. This model was trained without them. The GGUF ships with the trimmed template in its metadata instead of the dated one, so anything that reads the template from the file gets the format the weights expect.

In llama-cpp-python both of these are an exact match for training:

from llama_cpp import Llama

# no chat_format: use the template stored in the file
llm = Llama(model_path="akikawa-3.3-1b-roleplay_q8_0.gguf",
            n_ctx=4096, n_threads=6, verbose=False)

# or the library's built-in llama-3 formatter, which renders the same shape
llm = Llama(model_path="akikawa-3.3-1b-roleplay_q8_0.gguf",
            chat_format="llama-3",
            n_ctx=4096, n_threads=6, verbose=False)

The examples in this card pass chat_format="llama-3" so the choice is visible rather than implicit. llama-cli -cnv reads the embedded template, so it matches training out of the box.

Avoid anything that puts Meta's dated template back: an old Modelfile, a loader that ships its own Llama-3 Jinja, or a hand-written prompt that copies the stock chat format. The model never saw those two extra lines at the start of a system turn, so you would not be prompting the model this card describes. It will still answer, so the mismatch is easy to miss.

Do not use ChatML with this model. Markers it never saw in training, such as <|im_end|>, are not single tokens in the Llama-3 vocabulary: the tokenizer splits <|im_end|> into six ordinary tokens. The model then never receives a turn boundary it recognizes, so it keeps writing past the end of its reply and often answers as the Trainer too.

Using Transformers

The repo also ships the merged weights as model.safetensors, next to config.json, the tokenizer and chat_template.jinja. That is the simplest way in, because nothing gets dequantized on load:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

REPO = "Green-Eye/Akikawa-3.3-1b-Roleplay-128K-GGUF"

model = AutoModelForCausalLM.from_pretrained(REPO, dtype=torch.bfloat16, device_map="auto")
tokenizer = AutoTokenizer.from_pretrained(REPO)

Then generate as in the example further down, and pass use_cache=True to model.generate. (dtype needs a recent transformers; on older versions the argument is called torch_dtype.)

transformers can also read a GGUF directly: pass the file name as gguf_file and it builds the config, the weights and the tokenizer out of the metadata in the file. It needs transformers>=4.51 and the gguf package.

pip install "transformers>=4.51" gguf accelerate sentencepiece

from pathlib import Path

from transformers import AutoModelForCausalLM, AutoTokenizer

REPO = "Green-Eye/Akikawa-3.3-1b-Roleplay-128K-GGUF"
GGUF = "akikawa-3.3-1b-roleplay_q8_0.gguf"

model = AutoModelForCausalLM.from_pretrained(REPO, gguf_file=GGUF, device_map="auto")
tokenizer = AutoTokenizer.from_pretrained(REPO, gguf_file=GGUF)

SYSTEM = Path("system_prompt.txt").read_text(encoding="utf-8")
messages = [
    {"role": "system", "content": SYSTEM},
    {"role": "user", "content": "Good morning, Director!"},
]

inputs = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, return_tensors="pt", return_dict=True
).to(model.device)

out = model.generate(
    **inputs,
    max_new_tokens=200,
    do_sample=True,
    temperature=0.7,
    top_p=0.9,
    top_k=50,
    repetition_penalty=1.1,
    pad_token_id=tokenizer.eos_token_id,
)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Three things worth knowing before you take this route:

• It dequantizes on load. For a Llama checkpoint the weights are unpacked into a dense fp16/bf16 model in memory, so q8_0 costs about 2.5 GB of RAM here rather than 1.3 GB, and generation is slower than llama.cpp at the same job. The fast path that keeps the packed weights (on Metal) currently covers a small set of architectures, and Llama is not one of them yet.
• Use model.safetensors or the f16 file if you want the exact merged weights. Both are the unquantized model (bf16 and f16 respectively), so nothing is reconstructed on the way in. For further training, start from model.safetensors.
• The chat template travels with the model. On the gguf_file route it is read from the GGUF metadata, and the repo's own chat_template.jinja carries the same one. Both are the shape the model was trained on: the trimmed Llama-3 template with no date preamble. Check it once with print(tokenizer.chat_template). If it comes back None, render the prompt by hand in the shape shown under Chat format. Don't take the tokenizer from the base model's repository instead: that one carries Meta's dated template, and this model never saw those two lines.

For streaming, TextIteratorStreamer is the equivalent of stream=True above. It has to run in a thread, because generate blocks:

import threading

from transformers import TextIteratorStreamer

streamer = TextIteratorStreamer(tokenizer, skip_prompt=True, skip_special_tokens=True)
thread = threading.Thread(
    target=model.generate,
    kwargs={**inputs, "max_new_tokens": 200, "do_sample": True, "temperature": 0.7,
            "top_p": 0.9, "streamer": streamer, "pad_token_id": tokenizer.eos_token_id},
)
thread.start()
for token in streamer:
    print(token, end="", flush=True)
thread.join()

If you don't want to write any Python, recent transformers versions can serve a GGUF over an OpenAI-compatible endpoint straight from the Hub, using the repo and file name:

transformers serve Green-Eye/Akikawa-3.3-1b-Roleplay-128K-GGUF:akikawa-3.3-1b-roleplay_q8_0.gguf

How it was trained

The dataset is synthetic rather than scraped. A larger teacher model (Llama-3.3-8B-Instruct, 128K context) was given the lore profile and around 150 scenario templates covering casual encounters at the academy, official business, emotional beats and romance, and wrote multi-turn conversations in character. Each one was rendered under the full prompt shown further up (card, grounding guidelines and whatever remembered lines the conversation carried), which is why the model answers that framing so well and wanders when it is rewritten.

Every generated line then passed a quality gate that rejects truncated text, role bleed (a reply that starts writing the Trainer's next line), leaked instructions and unbalanced *action* markers. What survived was paraphrased, expanded and mood-shifted for variety, then deduplicated.

Only the assistant turns are supervised. The system prompt and the user turns are masked out with -100, so the model learns to speak as Akikawa instead of learning to reproduce the card.

Limitations

English only. The training data is entirely English.

It is a 1B model. It is good at voice and tone, and unreliable at facts, arithmetic and long chains of reasoning. It will invent details about the Uma Musume setting with total confidence.

One character. It is not a general assistant. Asked to be one, it answers in character anyway.

The romance is baked in. The card states an established relationship with the Trainer. Remove that line from the system prompt for a neutral dynamic, though the fine-tune will still lean warm.

Recall inside a conversation is weak. Tell it a name in turn 1 and ask for it in turn 5, and it may answer with a character from its own lore instead. If your application needs facts remembered, store them yourself and inject them into the system prompt rather than trusting the transcript.

It will not refuse you. The base model's refusals were removed with weight edits, and this fine-tune does not bring them back. There is no safety filtering on top, so it will go along with requests it probably shouldn't. That was the reason for picking this base, and it is also the risk. See Responsible use.

Responsible use

The wording here is the base model's, and it applies to this fine-tune unchanged:

Refusal suppression is deliberate and works as intended: this model will comply with requests the base model would refuse, including some it shouldn't. There is no safety filtering layered on top. You are responsible for how you deploy it, don't put this behind an unmoderated public-facing endpoint serving third parties. It inherits Llama-3.2-1B-Instruct's factual limitations and biases; abliteration removes refusal directions, it doesn't add capability or judgment.

• saidutta69/Llama-3.2-1B-Instruct-heretic

Two things specific to this repo. First, it plays a character, so everything it says comes in a voice. Warmth, opinions and confidence belong to the persona and say nothing about whether the content is true. Second, it is a 1B model trained on synthetic data. It makes up details about Uma Musume and about you, and it never knows when it is wrong. Don't put it in front of anyone who might believe an invented fact, and don't present it as a general-purpose assistant.

Built with Llama.

Llama 3.2 is licensed under the Llama 3.2 Community License, Copyright © Meta Platforms, Inc. All Rights Reserved.

This model derives from Llama-3.2-1B-Instruct-heretic, which derives from Llama-3.2-1B-Instruct. Both steps fall under the Llama 3.2 Community License, and any use of the model must also follow Meta's Acceptable Use Policy. Thanks to the author of the base model and to Heretic for the tool that produced it.

Fan project, not affiliated with anyone. Uma Musume: Pretty Derby and Yayoi Akikawa belong to Cygames.