PLaMo-3-610M-Fin-Instruct
Model Description
PLaMo-3-610M-Fin-Instruct is a Japanese language model for the financial domain, developed by Preferred Networks, Inc. It is fine-tuned from a PLaMo-3 base model through continued pre-training on financial corpora followed by supervised fine-tuning and direct preference optimization. This is a small instruction-tuned model for the financial domain. The model follows the PLaMo-3 chat format.
Usage
Requirements
python>=3.10.0
numpy>=1.26.4
numba>=0.60.0
torch>=2.6.0,<=2.9.0
transformers>=5.0.0
accelerate
Direct loading
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
model_name = "pfnet/plamo-3-610m-fin-instruct"
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
model_name, trust_remote_code=True, dtype=torch.bfloat16, device_map="auto"
)
messages = [
{"role": "user", "content": "PERとは何ですか?一文で答えて。"},
]
inputs = tokenizer.apply_chat_template(
messages, add_generation_prompt=True, return_dict=True, return_tensors="pt"
).to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512, do_sample=False)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))
The response contains a reasoning block delimited by <|plamo:begin_think:plamo|> and
<|plamo:end_think:plamo|>, followed by the answer, terminated by <|plamo:tag|>.
Model Details
- Developed by: Preferred Networks, Inc.
- Model type: Causal decoder-only
- Context length: 262,144 tokens
- Language(s): Japanese
- Tensor type: BF16
- License: PLaMo community license
Tokenizer
The tokenizer is the PLaMo-3 tokenizer, distributed with this repository as
tokenizer.jsonl and tokenization_plamo.py.
Bias, Risks, and Limitations
PLaMo-3-610M-Fin-Instruct is a new technology that carries risks with use. Testing conducted to date has been in Japanese, and has not covered, nor could it cover all scenarios. For these reasons, as with all LLMs, PLaMo-3-610M-Fin-Instruct's potential outputs cannot be predicted in advance, and the model may in some instances produce inaccurate, biased or other objectionable responses to user prompts. Therefore, before deploying any applications of PLaMo-3-610M-Fin-Instruct, developers should perform safety testing and tuning tailored to their specific applications of the model.
Domain specialization
This model has been additionally trained on financial-domain data and is intended for financial use cases. It is not a general-purpose assistant. If you intend to use it outside the financial domain, evaluate it on your own task first.
Model size and hallucination
This is a small model. It holds less factual knowledge than larger models and is more prone to hallucination: fluent, confident text that is factually wrong. Verify factual and numerical claims against an authoritative source before acting on them, and keep a human reviewer in the loop for consequential decisions.
Harmful content
Safety alignment is less reliable at this model size, and the model may generate harmful content, including offensive or violent expressions. Developers should apply content filtering appropriate to their deployment.
Not professional advice
The outputs of this model do not constitute legal, financial, tax, or investment advice, and must not be presented to end users as such. Applications built on this model should not be used to make investment decisions on a user's behalf.
License
PLaMo-3-610M-Fin-Instruct is released under the PLaMo Community License.
- (EN) https://plamo.preferredai.jp/info/plamo-community-license-en
- (JA) https://plamo.preferredai.jp/info/plamo-community-license-ja
Please review the PLaMo Community License and contact us via this form for commercial use: https://forms.gle/mTL8tBLrMYXKNZD56
Contributors
Preferred Networks, Inc.
- Ryo Wakai
- Ryota Ozaki
- Kentaro Imajo
- Masanori Hirano
- Lihang Hong
- Yoshiki Sato