NOTE: This "delta model" cannot be used directly.
Users have to apply it on top of the original LLaMA weights to get actual vicuna-13b-finetuned-langchain-MRKL weights.
See https://github.com/rinnakk/vicuna-13b-delta-finetuned-langchain-MRKL#model-weights for instructions.
vicuna-13b-finetuned-langchain-MRKL
Model details
Model type: vicuna-13b-finetuned-langchain-MRKL is an open-source chatbot trained by fine-tuning vicuna-13b on 15 examples with langchain-MRKL format.
Model Usage:
To obtain the correct model, plese run apply_delta.py first.(https://github.com/rinnakk/vicuna-13b-delta-finetuned-langchain-MRKL/blob/main/model/apply_delta.py) See instructions https://github.com/rinnakk/vicuna-13b-delta-finetuned-langchain-MRKL#model-weights
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("path/to/vicuna-13b-finetuned-langchain-MRKL")
model = AutoModelForCausalLM.from_pretrained("path/to/vicuna-13b-finetuned-langchain-MRKL")
model.cuda()
prompt = """Answer the following questions as best you can. You have access to the following tools:
Search: useful for when you need to answer questions about current events
Calculator: useful for when you need to answer questions about math
Use the following format:
Question: the input question you must answer
Thought: you should always think about what to do
Action: the action to take, should be one of [Search, Calculator]
Action Input: the input to the action
Observation: the result of the action
... (this Thought/Action/Action Input/Observation can repeat N times)
Thought: I now know the final answer
Final Answer: the final answer to the original input question
Begin!
Question: The current age of the President of the United States multiplied by 0.5.
Thought:"""
input_ids = tokenizer(prompt, return_tensors='pt').input_ids.to("cuda")
tokens = model.generate(input_ids,min_length = 5, max_new_tokens=128,do_sample = True, temperature = 0.7, top_p = 0.9)
print(tokenizer.decode(tokens[0], skip_special_tokens=True))
output(The tokens after "Thought:"):
I need to find the current age of the President and then multiply it by 0.5
Action: Search
Action Input: Who is the President of the United States?
if you launched a httpserver with the model and installed langchain(https://github.com/hwchase17/langchain), you can modify demo.py with your httpserver's ip&port and your SERPAPI_API_KEY, then run it.(https://github.com/rinnakk/vicuna-13b-delta-finetuned-langchain-MRKL/blob/main/demo.py) you can also try this by Jupyter Notebook. https://github.com/rinnakk/vicuna-13b-delta-finetuned-langchain-MRKL/blob/main/demo.ipynb
Where to send questions or comments about the model:
https://github.com/rinnakk/vicuna-13b-delta-finetuned-langchain-MRKL/issues
Training dataset
train only one epoch on mix data (sharegpt + 32*my.json + moss-003-sft-data)
Evaluation
- demo for langchain-MRKL: https://github.com/rinnakk/vicuna-13b-delta-finetuned-langchain-MRKL/blob/main/demo.ipynb
- No evaluation set. Because we don't think we improved the ability of model. we just make model fit langchain-MRKL strictly.
- We just want to show vicuna-13b's powerful ability about thinking and action.
Major Improvement
- support langchain-MRKL(agent= "zero-shot-react-description")
- very fast because of stritcly format(it doesn't generate redundant tokens)
Author
Qu Peng (https://huggingface.co/PengQu)