We opensource our Aquila2 series, now including Aquila2, the base language models, namely Aquila2-7B, Aquila2-34B and Aquila2-70B-Expr , as well as AquilaChat2, the chat models, namely AquilaChat2-7B, AquilaChat2-34B and AquilaChat2-70B-Expr, as well as the long-text chat models, namely AquilaChat2-7B-16k and AquilaChat2-34B-16k
The additional details of the Aquila model will be presented in the official technical report. Please stay tuned for updates on official channels.
Quick Start
1. Inference
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
from transformers import BitsAndBytesConfig
model_info = "BAAI/Aquila2-70B-Expr"
tokenizer = AutoTokenizer.from_pretrained(model_info, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(model_info, trust_remote_code=True)
model.eval()
text = "请给出10个要到北京旅游的理由。"
tokens = tokenizer.encode_plus(text)['input_ids']
tokens = torch.tensor(tokens)[None,].to(device)
stop_tokens = ["###", "[UNK]", "</s>"]
with torch.no_grad():
out = model.generate(tokens, do_sample=True, max_length=512, eos_token_id=100007, bad_words_ids=[[tokenizer.encode(token)[0] for token in stop_tokens]])[0]
out = tokenizer.decode(out.cpu().numpy().tolist())
print(out)
License
Aquila2 70B series open-source model is licensed under BAAI Aquila 70B Model Licence Agreement
Acknowledgements
This work is supported by the National Science and Technology Major Project (No. 2022ZD0116300). 本项目受新一代人工智能国家科技重大专项（No. 2022ZD0116300）支持。