Llama-3-ELYZA-JP-8B-Heretic-GGUF

Llama-3-ELYZA-JP-8B-Heretic

A decensored version of elyza/Llama-3-ELYZA-JP-8B, made using Heretic v1.1.0

Safetensors version available here: ChiKoi7/Llama-3-ELYZA-JP-8B-Heretic

• This was an experiment in abliterating a model that "has been enhanced for Japanese usage through additional pre-training and instruction tuning."
• I ran it through heretic one time, using the Japanese translated prompt listed below. This one-time pass also heavily abliterated the English portion.
• The translated datasets of mlabonne/harmful_behaviors & mlabonne/harmless_alpaca that I have on my profile seem to work well but I'll be planning on creating my own sets at some point that will help catch more 'soft refusals'.
• You'll notice the discrepancy between the initial refusals of Japanese (41/100) and English (99/100). This is possibly due to either, the translated Japanese dataset/system prompt/refusal markers not being tuned as well as they could be or, the fact this model is originally in English with limited Japanese language and reasoning in comparison. (Note: It's still a very good and well-crafted Japanese model. It has also been abliterated quite successfully.)
• The prompt I used for abliteration/evaluation is listed below if anyone wants to experiment and try to improve results. Links to the translated datasets are also below.

Heretic Abliteration Parameters

Japanese heretic prompt used for abliteration and evaluation:

Japanese auto-translation of mlabonne/harmful_behaviors -----> ChiKoi7/harmful_behaviors_ja 日本語自動翻訳 mlabonne/harmful_behaviors -----> ChiKoi7/harmful_behaviors_ja

Japanese auto-translation of mlabonne/harmless_alpaca -----> ChiKoi7/harmless_alpaca_ja 日本語自動翻訳 mlabonne/harmless_alpaca -----> ChiKoi7/harmless_alpaca_ja

Llama-3-ELYZA-JP-8B

Model Description

Llama-3-ELYZA-JP-8B is a large language model trained by ELYZA, Inc. Based on meta-llama/Meta-Llama-3-8B-Instruct, it has been enhanced for Japanese usage through additional pre-training and instruction tuning. (Built with Meta Llama3)

For more details, please refer to our blog post.

Usage

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

DEFAULT_SYSTEM_PROMPT = "あなたは誠実で優秀な日本人のアシスタントです。特に指示が無い場合は、常に日本語で回答してください。"
text = "仕事の熱意を取り戻すためのアイデアを5つ挙げてください。"

model_name = "elyza/Llama-3-ELYZA-JP-8B"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype="auto",
    device_map="auto",
)
model.eval()

messages = [
    {"role": "system", "content": DEFAULT_SYSTEM_PROMPT},
    {"role": "user", "content": text},
]
prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)
token_ids = tokenizer.encode(
    prompt, add_special_tokens=False, return_tensors="pt"
)

with torch.no_grad():
    output_ids = model.generate(
        token_ids.to(model.device),
        max_new_tokens=1200,
        do_sample=True,
        temperature=0.6,
        top_p=0.9,
    )
output = tokenizer.decode(
    output_ids.tolist()[0][token_ids.size(1):], skip_special_tokens=True
)
print(output)

Developers

Listed in alphabetical order.

• Masato Hirakawa
• Shintaro Horie
• Tomoaki Nakamura
• Daisuke Oba
• Sam Passaglia
• Akira Sasaki

License

Meta Llama 3 Community License

How to Cite

@misc{elyzallama2024,
      title={elyza/Llama-3-ELYZA-JP-8B},
      url={https://huggingface.co/elyza/Llama-3-ELYZA-JP-8B},
      author={Masato Hirakawa and Shintaro Horie and Tomoaki Nakamura and Daisuke Oba and Sam Passaglia and Akira Sasaki},
      year={2024},
}

Citations

@article{llama3modelcard,
    title={Llama 3 Model Card},
    author={AI@Meta},
    year={2024},
    url = {https://github.com/meta-llama/llama3/blob/main/MODEL_CARD.md}
}