LOOP DFS Qwen Merged 16-bit Model
Revised README.md
# LOOP DFS Qwen Merged 16-bit Model
This repository contains a merged 16-bit Qwen model fine-tuned for LOOP DFS assistance.
The model is designed to answer questions about LOOP DFS company information, products, services, partnerships, and documented API behavior.
## Generation configuration
The recommended generation settings are:
- `max_new_tokens`: 512
- `temperature`: 0.2
- `top_p`: 0.9
- `repetition_penalty`: 1.05
The complete system prompt is stored in `system_prompt.txt`.
Machine-readable inference settings are stored in `deployment_config.json`.
## Quick start
Install the required packages:
```bash
pip install -U transformers accelerate huggingface_hub sentencepiece torch
```
Load the model and its system prompt directly from this public repository:
```python
import torch
from huggingface_hub import hf_hub_download
from transformers import AutoModelForCausalLM, AutoTokenizer
MODEL_ID = "RayNene/Loop-DFS-Qwen-Merged"
MAX_NEW_TOKENS = 512
# Load the complete system prompt from the repository.
system_prompt_path = hf_hub_download(
repo_id=MODEL_ID,
filename="system_prompt.txt",
)
with open(system_prompt_path, "r", encoding="utf-8") as file:
SYSTEM_PROMPT = file.read().strip()
if not SYSTEM_PROMPT:
raise RuntimeError("system_prompt.txt is empty.")
# Select an appropriate inference data type.
if torch.cuda.is_available():
inference_dtype = (
torch.bfloat16
if torch.cuda.is_bf16_supported()
else torch.float16
)
else:
inference_dtype = torch.float32
# Load the tokenizer.
tokenizer = AutoTokenizer.from_pretrained(
MODEL_ID,
trust_remote_code=True,
)
if tokenizer.pad_token_id is None:
tokenizer.pad_token = tokenizer.eos_token
# Load the merged model.
model = AutoModelForCausalLM.from_pretrained(
MODEL_ID,
torch_dtype=inference_dtype,
device_map="auto",
trust_remote_code=True,
low_cpu_mem_usage=True,
)
model.eval()
# Configure valid end-of-response tokens.
terminator_ids = []
for token_id in [
tokenizer.eos_token_id,
tokenizer.convert_tokens_to_ids("<|im_end|>"),
]:
if token_id is None:
continue
if token_id == tokenizer.unk_token_id:
continue
if token_id not in terminator_ids:
terminator_ids.append(token_id)
if not terminator_ids:
raise RuntimeError("No valid generation terminator was found.")
def clean_answer(answer):
"""Prevent accidental generation of another conversation turn."""
stop_markers = [
"<|im_end|>",
"<|im_start|>system",
"<|im_start|>user",
"<|im_start|>assistant",
"\nSystem:",
"\nUser:",
]
positions = [
answer.find(marker)
for marker in stop_markers
if answer.find(marker) >= 0
]
if positions:
answer = answer[:min(positions)]
return answer.strip()
def ask(question, history=None):
"""Generate one LOOP DFS assistant response."""
if not isinstance(question, str) or not question.strip():
raise ValueError("The question must be a non-empty string.")
if history is None:
history = []
messages = [
{
"role": "system",
"content": SYSTEM_PROMPT,
},
*history,
{
"role": "user",
"content": question.strip(),
},
]
prompt = tokenizer.apply_chat_template(
messages,
tokenize=False,
add_generation_prompt=True,
)
inputs = tokenizer(
prompt,
return_tensors="pt",
add_special_tokens=False,
)
input_device = model.get_input_embeddings().weight.device
inputs = {
name: tensor.to(input_device)
for name, tensor in inputs.items()
}
prompt_length = inputs["input_ids"].shape[1]
with torch.inference_mode():
output = model.generate(
**inputs,
max_new_tokens=MAX_NEW_TOKENS,
do_sample=True,
temperature=0.2,
top_p=0.9,
repetition_penalty=1.05,
eos_token_id=terminator_ids,
pad_token_id=tokenizer.pad_token_id,
use_cache=True,
)
generated_tokens = output[0, prompt_length:]
answer = tokenizer.decode(
generated_tokens,
skip_special_tokens=True,
)
return clean_answer(answer)
answer = ask("Who is the CEO of LOOP DFS Kenya?")
print(answer)
```
## Interactive chat
The following starter code maintains conversation history and uses the complete system prompt loaded from `system_prompt.txt`:
```python
conversation = []
print("LOOP DFS Assistant")
print("Commands: clear | exit | quit")
print()
while True:
try:
user_message = input("You: ").strip()
if not user_message:
continue
command = user_message.lower()
if command in {"exit", "quit"}:
print("Chat ended.")
break
if command == "clear":
conversation.clear()
print("Conversation history cleared.")
print()
continue
response = ask(
question=user_message,
history=conversation,
)
print(f"\nLOOP DFS Assistant: {response}\n")
conversation.extend(
[
{
"role": "user",
"content": user_message,
},
{
"role": "assistant",
"content": response,
},
]
)
except KeyboardInterrupt:
print("\nChat ended.")
break
```
## Google Colab starter
In a new Google Colab notebook:
1. Select **Runtime → Change runtime type → GPU**.
2. Run the installation cell:
```python
!pip -q install -U transformers accelerate huggingface_hub sentencepiece
```
3. Run the Python code from the **Quick start** section.
4. Run the **Interactive chat** section.
No Hugging Face token is required because the repository is public.
## Loading the deployment configuration
Applications can load the recommended generation configuration directly from `deployment_config.json`:
```python
import json
from huggingface_hub import hf_hub_download
config_path = hf_hub_download(
repo_id="RayNene/Loop-DFS-Qwen-Merged",
filename="deployment_config.json",
)
with open(config_path, "r", encoding="utf-8") as file:
deployment_config = json.load(file)
print(deployment_config)
```
It can then be applied during generation:
```python
output = model.generate(
**inputs,
max_new_tokens=deployment_config.get(
"max_new_tokens",
512,
),
do_sample=deployment_config.get(
"do_sample",
True,
),
temperature=deployment_config.get(
"temperature",
0.2,
),
top_p=deployment_config.get(
"top_p",
0.9,
),
repetition_penalty=deployment_config.get(
"repetition_penalty",
1.05,
),
eos_token_id=terminator_ids,
pad_token_id=tokenizer.pad_token_id,
use_cache=True,
)
```
## Deployment requirements
Current company facts, product terms, fees, eligibility requirements, leadership information, partnerships, and API behavior should be supplied through approved retrieval sources or verified tools.
Authentication and authorization must be handled outside the conversation. The assistant must never collect passwords, PINs, OTPs, CVVs, secret keys, access tokens, or other authentication credentials through chat.
Financial transactions must be validated by application code, summarized for the user, and explicitly confirmed through an approved secure flow. The model must not be used as the transaction authorization layer.
## Repository files
- `system_prompt.txt` — complete production system prompt
- `deployment_config.json` — recommended generation and deployment settings
- `config.json` — model architecture configuration
- `generation_config.json` — model generation configuration, when included
- `tokenizer_config.json` — tokenizer configuration
- `tokenizer.json` — tokenizer vocabulary and rules, when included
- `*.safetensors` — merged model weights