FinMA-7B-NLP
📄 Paper · 💻 Code · 🌐 The Fin AI
Part of PIXIU — PIXIU: A Large Language Model, Instruction Data and Evaluation Benchmark for Finance (arXiv:2306.05443).
FinMA-7B-NLP is a financial large language model (LLM) developed as part of the PIXIU project. It is designed to understand complex financial language and concepts, and is fine-tuned to follow natural language instructions, enhancing its performance in downstream financial tasks. Specifically, FinMA-7B-NLP is trained only on the NLP tasks of the PIXIU dataset, making it specialized for tasks such as sentiment analysis, news headline classification, named entity recognition, and question answering.
Other Models in the PIXIU Project
In addition to FinMA-7B-NLP, the PIXIU project includes two other models: FinMA-7B-full and FinMA-30B.
-
FinMA-7B-full: This model is trained with the full instruction data from the PIXIU dataset, covering both NLP and prediction tasks. This makes it a more comprehensive model capable of handling a wider range of financial tasks.
-
FinMA-30B: This model is a larger version of FinMA, fine-tuned on the LLaMA-30B model. Like FinMA-7B-NLP, it is trained with the NLP instruction data.
Usage
You can use the FinMA-7B-NLP model in your Python project with the Hugging Face Transformers library. Here is a simple example of how to load the model:
from transformers import LlamaTokenizer, LlamaForCausalLM
tokenizer = LlamaTokenizer.from_pretrained('ChanceFocus/finma-7b-nlp')
model = LlamaForCausalLM.from_pretrained('ChanceFocus/finma-7b-nlp', device_map='auto')
In this example, LlamaTokenizer is used to load the tokenizer, and LlamaForCausalLM is used to load the model. The device_map='auto' argument is used to automatically use the GPU if it's available.
Hosted Inference API
You can also use the model through the Hugging Face Inference API. This allows you to generate text without having to set up your own inference environment. The model can be loaded on the Inference API on-demand.
License
FinMA-7B-NLP is licensed under MIT. For more details, please see the MIT file.
About
This model is part of the PIXIU project, an open-source resource featuring the first financial large language models (LLMs), instruction tuning data, and evaluation benchmarks to holistically assess financial LLMs. The goal is to continually push forward the open-source development of financial artificial intelligence (AI).
For more information, you can visit the PIXIU project on GitHub.
Citation
If you use FinMA-7B-NLP in your work, please cite the PIXIU paper:
@misc{xie2023pixiu,
title={PIXIU: A Large Language Model, Instruction Data and Evaluation Benchmark for Finance},
author={Qianqian Xie and Weiguang Han and Xiao Zhang and Yanzhao Lai and Min Peng and Alejandro Lopez-Lira and Jimin Huang},
year={2023},
eprint={2306.05443},
archivePrefix={arXiv},
primaryClass={cs.CL}
}