llm-jp-3.1-1.8b-v02b

Japanese-focused fine-tuned derivative of llm-jp/llm-jp-3.1-1.8b.

This is an unofficial community derivative and is not an official LLM-jp release.

Important compatibility note

The originally published GGUF build has been removed because its tokenizer conversion was incorrect.

The base LLM-jp 3.1 tokenizer is a Hugging Face tokenizers Unigram tokenizer with byte fallback. Converting it as a standard llama tokenizer produced tokenization mismatches.

A corrected UGM-based GGUF build is now provided separately. It reproduces the Hugging Face tokenizer on the tested prompt, but currently requires a llama.cpp runtime with the relevant UGM byte-fallback fix applied.

The corrected UGM GGUF is not currently compatible with the standard LM Studio runtime tested by the author. In that runtime, generation terminated immediately with EOS.

For the most reliable use today, use the Hugging Face merged model with Transformers.

Files

Hugging Face merged model

The repository root contains the merged Hugging Face model files.

This is the recommended build.

from transformers import AutoTokenizer, AutoModelForCausalLM

repo_id = "ago99/llm-jp-3.1-1.8b-v02b-GGUF"

tokenizer = AutoTokenizer.from_pretrained(repo_id)

model = AutoModelForCausalLM.from_pretrained(
    repo_id,
    torch_dtype="auto",
    device_map="auto"
)

Experimental UGM GGUF

v02b-ugm-Q4_K_M.gguf

• Quantization: Q4_K_M
• Tokenizer model: UGM / t5
• Token count: 99,584
• Context length: 4096
• Requires a llama.cpp runtime with the relevant UGM byte-fallback fix
• Not currently compatible with the standard LM Studio runtime tested by the author

The UGM conversion was validated against the Hugging Face tokenizer on a full instruction prompt:

• Hugging Face token count: 42
• UGM token count: 42
• Token IDs: exact match

With the patched runtime, both F16 GGUF and Q4_K_M GGUF passed the same 5/5 sanity test.

Model

• Base model: llm-jp/llm-jp-3.1-1.8b
• Parameters: approximately 1.8B
• Primary language: Japanese
• Context length: 4096
• Intended use: lightweight local Japanese conversation and instruction following

What was changed

The base model was further trained and tuned with the goal of improving:

• Japanese instruction following
• concise Japanese responses
• reading and semantic understanding
• response stability
• medium/long-form generation behavior

The final v0.2b LoRA adapter was merged into the v0.2a fine-tuned model at scale 1.00.

Prompt format

The model was trained and evaluated with the following instruction format:

以下は、タスクを説明する指示です。要求を適切に満たす応答を書きなさい。

### 指示:
<user prompt>

### 応答:

For direct Transformers inference, the tokenizer was used with add_special_tokens=False for this formatted prompt.

Evaluation notes

Internal evaluation of the final merged model before GGUF quantization showed approximately:

• Stress test: 180 / 200
• Knowledge test: 257 / 300
• Strict instruction test: 16 / 20
• Length control: 10 / 10
• Long-form collapse: approximately 1 / 10

A later 10-question sanity check on the merged Hugging Face model scored 9 / 10.

These are internal experimental measurements and should not be interpreted as standardized benchmark scores.

GGUF tokenizer investigation

During GGUF validation, the original conversion path was found to tokenize the same prompt differently from the Hugging Face tokenizer.

Example:

Hugging Face Unigram:

なさい / 。

Original GGUF llama tokenizer:

なさ / い。

A corrected UGM conversion produced exact token-ID agreement for the tested full prompt.

The corrected UGM GGUF therefore replaces the earlier GGUF conversion, but runtime support is still required for correct operation.

Known limitations

This is a small approximately 1.8B parameter model.

Known limitations include:

• Hallucinations can occur, especially for fictitious or unknown entities.
• Factual accuracy is limited compared with larger models.
• Long explanations may contain factual errors.
• The model may confidently generate incorrect information.
• The experimental UGM GGUF currently depends on runtime support that is not yet universal.
• Standard LM Studio runtime compatibility is not currently available in the author's test environment.

The model should not be relied upon for medical, legal, financial, safety-critical, or other high-stakes decisions.

For factual or current information, external retrieval or verification is recommended.

Example

Prompt:

日本の首都はどこですか？答えだけ。

Example output:

東京

Base model

This model is based on:

llm-jp/llm-jp-3.1-1.8b

Developed by the Research and Development Center for Large Language Models at the National Institute of Informatics (NII).

License

The original model is distributed under the Apache License 2.0.

This derivative model is also distributed under the Apache License 2.0.

See the LICENSE file in this repository for the full license text.

This model has been modified from the original LLM-jp model through additional training, LoRA fine-tuning, model merging, GGUF conversion, and quantization.

The original LLM-jp project and National Institute of Informatics do not endorse or certify this derivative model.

Disclaimer

This is an experimental community model.

Outputs may be inaccurate, fabricated, biased, incomplete, or otherwise unreliable.

Users are responsible for evaluating whether the model is appropriate for their intended use.