my-qwen-lora-cisco
A merged (LoRA-finetuned) variant of Qwen/Qwen2.5-1.5B-Instruct, fine-tuned for Cisco-related assistant tasks.
This repository contains full merged weights — not a PEFT adapter. Load it directly with
transformers; no peft and no adapter merging step is required.
Model Details
| Base model | Qwen/Qwen2.5-1.5B-Instruct |
| Architecture | Qwen2ForCausalLM (qwen2) |
| Parameters | 1,543.7M (~1.5B) |
| Precision | bfloat16 |
| Context length | 32,768 tokens |
| Vocab size | 151,936 |
| Chat template | Included (chat_template.jinja, ChatML) |
| License | Apache 2.0 (inherited from the base model) |
Quickstart
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
model_id = "banhcarrot/my-qwen-lora-cisco"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id, torch_dtype=torch.bfloat16, device_map="auto"
)
messages = [{"role": "user", "content": "Hello, what can you help me with?"}]
inputs = tokenizer.apply_chat_template(
messages, add_generation_prompt=True, return_tensors="pt"
).to(model.device)
outputs = model.generate(
inputs, max_new_tokens=256, temperature=0.7, top_p=0.8, top_k=20,
repetition_penalty=1.1, do_sample=True,
)
print(tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True))
Serving it
# vLLM (OpenAI-compatible server)
vllm serve banhcarrot/my-qwen-lora-cisco
# TGI
text-generation-launcher --model-id banhcarrot/my-qwen-lora-cisco
The model is tagged endpoints_compatible and can be deployed directly to
Hugging Face Inference Endpoints
for a dedicated, OpenAI-compatible /v1/chat/completions API.
Training
- Method: LoRA / QLoRA fine-tuning on top of Qwen2.5-1.5B-Instruct, then merged into the base weights
- Training data: [describe your dataset — size, source, language, and whether it is public]
- Objective: [e.g. supervised fine-tuning on Cisco-domain Q&A]
- Hardware: [e.g. 1× T4 / Colab / local GPU]
- LoRA config: [rank
r, alpha, target modules, epochs, LR] - Framework: [e.g. TRL
SFTTrainer/ Unsloth / PEFT]
Intended Use
Direct use: [e.g. answering Cisco product, configuration, and troubleshooting questions.]
Out of scope: This model is not production-grade for safety- or compliance-critical networking decisions. Verify all configuration and command output before applying it to real hardware. Do not use it as the sole source of authority for security-sensitive changes.
Limitations
- A 1.5B model — expect weaker reasoning than larger Qwen2.5 variants and occasional factual errors about specific Cisco SKUs, IOS versions, or licensing.
- Fine-tuning may have narrowed the model's general-purpose abilities compared to the base model.
- No guarantee of accuracy for time-sensitive information; the base model has a knowledge cutoff.
- Bias, risks, and limitations of the base model still apply.
- No safety alignment work was performed beyond what the base Instruct model provides.
Bias, Risks, and Ethical Considerations
Fine-tuning on a domain-specific dataset can amplify any biases present in that data. [Add anything you know about your training data's provenance, licensing, and contents.]
Citation
@misc{banhcarrot_my_qwen_lora_cisco,
title = {my-qwen-lora-cisco},
author = {banhcarrot},
year = {2026},
url = {https://huggingface.co/banhcarrot/my-qwen-lora-cisco}
}