Index-Translate-2B-JP-Manga-Finetune-v1

Index-Translate-2B fine-tuned for Japanese → English translation of manga dialogue. It is the counterpart of Hy-MT2-1.8B-JP-Manga-Finetune-v5 on a newer base model, trained on the same manga corpus. This repo carries both the Q4_K_M GGUF and the merged bf16 weights.

Slight improvement compared with Hy-MT2 v5. Initial user testing indicates this model is a slight but noticeable improvement over the Hy-MT2 v5 finetune. On about 2,500 lines of held-out manga, with both models as Q4_K_M and greedy decoding, chrF++ against reference translations is 46.7 for this model and 45.2 for v5 (+1.5, 95% interval +0.7 to +2.3). COMET-22 is level (79.3 vs 79.1). It has not been through the blind pairwise judge quoted on the v5 card, so read this as "at least as good on automatic scores", not as a proven successor.

It costs a little more to run. The GGUF is 1.27 GB against 1.13 GB, and it generates about 10% fewer tokens per second than v5 on CPU in llama.cpp. Its architecture also changes how a client keeps the instruction cached between lines; see Reusing the prompt prefix. With that in place one line took about 230 ms on a Snapdragon 8 Elite phone, against about 210 ms for v5. Without it, about 500 ms.

It needs its own prompt. With the Hy-MT2 prompt it scores 44.3 chrF++, below both its own prompt and v5, so a client that sends the Hy-MT2 prompt will not get this model's quality. See Prompting. FumetoReaderPlus 0.7.4 and later offer it as an on-device model and apply the prompt, chat format and decoding described here by themselves; earlier versions only send the Hy-MT2 prompt.

Japanese → English, text only. The base model covers 150 languages and image input. This fine-tune was trained and tested on Japanese → English text alone; use the base model for anything else. Asked for German, Chinese, Korean or Spanish output, it still answered in English.

Scope

Built for manga: short lines, in the register of the source.

• One line at a time by default. The model does not see the page; where Japanese leaves the subject open it usually commits to a reading, about as often as v5 does. Page mode gives it the other lines on the page.
• Very short lines are the weak spot. On lines under 4 characters (interjections, sound effects) it scores below v5 (25.3 vs 27.6 chrF++). On everything longer it scores above.
• Coarse language comes out slightly softer than from v5, by a lexical check.
• Kana occasionally survives in a name. 2 of 2,553 test lines came back with a half-romanised name.
• Expect v5's other biases. They come from the training data, not the base model, and were not re-measured here: truncated input gets finished into a whole utterance, and real-world proper nouns and specialist vocabulary are probably weaker than in the base model.

Prompting

The instruction is in Chinese, following the base model's native prompt format, and the line stays in Japanese. Three layouts were trained. Other wordings are untested, and there is no system prompt.

One line

The default, and the layout the scores above were measured with.

请将以下日语漫画对白翻译为英语，直接输出翻译结果，不要进行任何解释。

<Japanese line>

For example, 先輩、部室の鍵どこに置きました？ returns Senpai, where did you leave the clubroom key?

For text that is not manga dialogue, the base model's wording 日语文本 in place of 日语漫画对白 was trained too, and scores the same on manga.

One line with a glossary

For names and terms you want rendered a particular way.

请将以下日语漫画对白翻译成英语，并且严格遵循所有约束要求。

【源文】
ミカ、生徒会室に来てくれ

【约束要求】
1. 【硬性要求】专名/术语对照: ミカ→Mika、生徒会室→student council room

只输出译文，不要有任何额外说明。

returns Mika, come to the student council room.

• Pairs are Japanese→English, joined with 、.
• List only pairs whose Japanese side occurs in the line. It was trained with one to three pairs per line, mostly one.
• When there is nothing to list, use the one-line layout instead.
• Treat it as a strong hint, not a guarantee. On a test that supplies a name the model would not choose by itself, it used the supplied one in 84% of 200 sentences (the base model: 93%).

A page at a time

All the lines of a page in one request, as a JSON object keyed "1" to "N" in reading order. 共4句 carries the number of lines.

请将以下日语漫画对白翻译成英语，共4句，严格遵循所有约束要求。

【待翻译文本】
{"1": "おはよう！", "2": "…眠い", "3": "また徹夜でゲームしてたの？", "4": "ち、違うって！勉強してたんだよ！"}

【约束要求】
1. 【硬性要求】保留源文中的 JSON 格式标记不变
2. 【注意】译文保持口语/随意语体风格

请以相同的 JSON 格式输出翻译结果，key 保持不变，value 为对应译文。只输出 JSON，不要有任何额外说明。
示例输出格式：{"1": "译文1", "2": "译文2", ...}

returns

{"1": "Morning!", "2": "...I'm sleepy.", "3": "Were you up all night playing games again?", "4": "N-no, that's not it! I was studying!"}

• It was trained with 2 to 8 lines per request.
• Keep a per-line fallback. Of 505 test pages (up to 22 lines each), 498 came back as a complete JSON object, 3 were missing lines and 4 did not parse.
• It scores slightly above one line at a time on the same test (47.2 vs 46.7 chrF++).

Chat format

ChatML with a single user turn. The answer starts directly after the assistant header and ends at <|im_end|>:

<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant

The chat template in this repo, also embedded in the GGUF, produces exactly that. It is not the base model's template. The base opens every answer with an empty <think> block, which this fine-tune was trained without; using the base template measured 0.4 chrF++ lower.

Sampling

Greedy: temperature 0, no repeat penalty. Every score on this card was measured that way, and the base model's card uses temperature 0 too.

Do not carry over the sampler recommended for the Hy-MT2 fine-tunes (temperature 0.15, top_k 20, top_p 0.6, repeat_penalty 1.05): it measured 0.6 chrF++ lower than greedy on this model.

A one-line request is 29 to 97 prompt tokens (median 35) and the answer is rarely over 40, so a 512-token context and max_tokens 128 are enough. For a page of up to 8 lines, a 1024-token context and max_tokens 512 are enough. Always set a token cap: on the rare line or page where the output goes wrong, it can run until the cap.

Reusing the prompt prefix

Every one-line request starts with the same 23 tokens (the chat header and the instruction) and a typical request is about 35, so a client that translates line after line wants to decode those 23 once.

With Hy-MT2 that is a KV-cache rewind: keep the prompt in the context, remove everything after the instruction, decode the new line. This model is a Qwen3.5 hybrid with recurrent layers between its attention layers, and a recurrent state cannot be rewound. llama_memory_seq_rm returns false for a removal that would cut into it, so a client that relies on the rewind decodes the whole prompt every time.

What works instead, through the llama.cpp C API:

1. Decode the 23 fixed tokens as their own batch.
2. Save the sequence's recurrent state: llama_state_seq_get_data_ext(ctx, buf, size, 0, LLAMA_STATE_SEQ_FLAGS_PARTIAL_ONLY). It is about 20 MB.
3. Decode the rest of the prompt and generate.
4. For each later line: llama_state_seq_set_data_ext with the saved buffer and the same flag, then llama_memory_seq_rm(mem, 0, 23, -1), which now succeeds, then decode from position 23.

Restoring takes a few milliseconds and is exact: the output is byte-identical to decoding the same two batches from an empty context. Decoding a prompt in two batches is not bit-identical to decoding it in one, though. About one line in five came out worded differently, and over the 2,553 test lines the two score the same (46.8 and 47.0 chrF++, not a significant difference).

In the llama-server build tested, cache_prompt did not shorten a request that only shares its prefix with the previous one, so through the server each line paid for its whole prompt.

Running it

llama.cpp

llama-server -m index-translate-2b-manga-v1-Q4_K_M.gguf -c 4096

curl http://127.0.0.1:8080/v1/chat/completions -H "Content-Type: application/json" -d '{
  "messages": [{"role": "user", "content": "请将以下日语漫画对白翻译为英语，直接输出翻译结果，不要进行任何解释。\n\n先輩、部室の鍵どこに置きました？"}],
  "temperature": 0, "max_tokens": 128
}'

transformers (tested with 5.6.0)

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "fumetodev/Index-Translate-2B-JP-Manga-Finetune-v1-GGUF"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, dtype=torch.bfloat16, device_map="auto")

def translate(ja: str) -> str:
    prompt = f"请将以下日语漫画对白翻译为英语，直接输出翻译结果，不要进行任何解释。\n\n{ja}"
    inputs = tok.apply_chat_template([{"role": "user", "content": prompt}], add_generation_prompt=True,
                                     return_tensors="pt", return_dict=True).to(model.device)
    out = model.generate(**inputs, max_new_tokens=128, do_sample=False)
    return tok.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True).strip()

print(translate("先輩、部室の鍵どこに置きました？"))

Files

• index-translate-2b-manga-v1-Q4_K_M.gguf — 1,274,389,024 bytes, sha256 7f8475eb80dce562fdf7c7283d78fd79275f5e5f19ffcd1118f0d7e91021b5d5. It has no multi-token-prediction block. The base model's official GGUF carries one, which an April 2026 llama.cpp build refuses to load; this file loads there and on a July 2026 build. <|im_end|> is an end-of-generation token, so no stop strings are needed.
• model.safetensors with config.json, the tokenizer files and chat_template.jinja — merged bf16 weights, Qwen3_5ForCausalLM, loadable with transformers without trust_remote_code. The tokenizer is the base model's; the chat template is not (see Chat format). The base model's vision tower is included unchanged, and its multi-token-prediction head is left out. If you convert to GGUF yourself, keep this repo's chat template.