PLaMo 3 12B Self-Merged Base
Important!: Currently, this model has not undergone CPT (Continued Pre-Training) following the merge; it will be replaced later with a CPT-processed version.
PLaMo 3 12B Self-Merged Base is a model created by self-merging pfnet/plamo-3-nict-8b-base and scaling it up to 12B parameters.
PLaMo 3 12B Self-Merged Base currently has some unstable elements, which we plan to address in an upcoming update.
PLaMo 3 12B Self-Merged Base is released under the PLaMo Community License. Please review the license below and agree to its terms before downloading.
- (EN) https://plamo.preferredai.jp/info/plamo-community-license-en
- (JA) https://plamo.preferredai.jp/info/plamo-community-license-ja
NOTE: This model has NOT been instruction-tuned for chat dialogue or other downstream tasks.
This is a merge of pre-trained language models created using mergekit.
For commercial use
Please review the PLaMo Community License and contact us using the form below if you intend to use this model for commercial purposes.
Usage
Requirements
python>=3.10.0
numpy>=1.26.4
numba>=0.60.0
torch>=2.6.0,<=2.9.0
transformers>=4.56.0,<=4.57.1
Loading the model directly
from transformers import AutoTokenizer, AutoModelForCausalLM
model_id = "midorin-Linux/plamo-3-12b-self-merged"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(model_id, trust_remote_code=True)
text = "これからの人工知能技術は"
inputs = tokenizer(text, return_tensors="pt").to(model.device)
generated_tokens = model.generate(
**inputs,
max_new_tokens=32,
do_sample=True,
top_k=50,
top_p=0.95,
temperature=1.0,
pad_token_id=tokenizer.eos_token_id,
use_cache=True,
)[0]
generated_text = tokenizer.decode(generated_tokens, skip_special_tokens=True)
print(generated_text)
Model Details
- Model size: 12B (8B + 8B/2)
- Base model (pfnet/plamo-3-nict-8b-base) developed by:
- Preferred Networks, Inc.
- In collaboration with the National Institute of Information and Communications Technology (NICT)
- Model type: Causal decoder-only
- Language(s): English, Japanese
- License: PLaMo Community License
Base Model Information
For details on the base model, please refer to pfnet/plamo-3-nict-8b-base.
Built with PLaMo