ActionGemma 2B: merged FP16 weights (vLLM / Transformers)
These are the LoRA adapter of ActionGemma 2B merged into the full Gemma 2 2B weights at 16-bit precision. They are ready for GPU serving with vLLM, or for plain Transformers without needing PEFT.
Compared with the base model, ActionGemma reaches 93.74% macro F1 (+7.82%), cuts hallucinated function calls from 16% to 6.7%, and uses about 17% fewer tokens. See the main model card for full results and training details.
Serve with vLLM
pip install vllm
vllm serve dinushiTJ/action-gemma-2-2b-it-vllm-f16
This starts an OpenAI-compatible API on http://localhost:8000/v1. Put the function-calling prompt (shown in the GGUF card) at the start of the first user message. Alternatively, build the prompt with the tokenizer's tool_use chat template, as shown in the main model card.
Load with Transformers
from transformers import AutoModelForCausalLM, AutoTokenizer
repo = "dinushiTJ/action-gemma-2-2b-it-vllm-f16"
tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, torch_dtype="auto", device_map="auto")
Related
- LoRA adapter and full model card:
dinushiTJ/action-gemma-2-2b-it-lora - GGUF for laptops and CPUs:
dinushiTJ/action-gemma-2-2b-it-gguf
Use of this model is subject to the Gemma Terms of Use. Author: Dinushi Jayasinghe.