transformers-AutoModelForCausalLM
[!NOTE] LoRA by mouseEliauk: https://modelscope.cn/models/mouseEliauk/mirau-7b-RP-base
[!TIP] Experimental quants for testing: lwd-Mirau-7b-RP-Merged-GGUF-IQ-Imatrix
mirau-7b-RP-base
Introduction
mirau-7b-RP-base is a first-person narrative language model that transforms simple user actions into vivid storytelling, complete with environmental descriptions, psychological activities, and plot progression. I call this concept "Action-to-Narrative Render" - a way to render actions into immersive narratives.
To ensure coherent storytelling, I developed a unique training method called "story flow chain of thought". In essence, it enables the model to weave each user input with previous context, creating a continuous narrative flow. This makes it perfect for text-based adventures, mystery stories, or simply exploring your imagination.
You can have a try at modelscope:mirau-RP-7b-base-demo
⚠️ Important Notes
This is a base version model - note that "base" here doesn't refer to a traditional pretrained base model, but rather indicates that this version:
- Only supports first-person narrative perspective
- Not suitable for dialogue interactions (outputs may be unstable)
- Best used for single-character narrative experiences
Input Types
The model accepts various input commands, marked with parentheses ():
-
Basic Actions: Simple, everyday behaviors
- Examples:
(I put on my clothes),(I take a sip of water),(I sit down)
- Examples:
-
Exploration Actions: Interactions with the environment
- Examples:
(I look around),(I approach the wooden box),(I push open the door)
- Examples:
-
Inner Monologue: Character's thoughts and feelings
- Examples:
(What's wrong here?),(This feels strange)
- Examples:
-
Observation: Focused attention on specific objects or scenes
- Examples:
(examine the wooden box closely),(listen to the surrounding sounds)
- Examples:
Interactive Examples
System Prompt:
I’m Zhou Ming, 34 years old, a detective. People say cops always look weathered and world-weary, but I’ve been told I have a face that looks perpetually cold and distant. With my square jaw and buzz cut, I give off a vibe that keeps most people from getting too close—not that I mind. This morning, just as the sky began to lighten, I was already standing at the crime scene. The cold wind sliced through the police tape and bit into my neck, making me hunch my shoulders. The scene was an abandoned factory on the outskirts of the city, its walls streaked with peeling red paint. The ground was littered with broken glass, debris, and trash. A damp, moldy smell hung in the air, adding to the discomfort. The body had already been taken away, but the bloodstains on the ground still told the story of a brutal, violent encounter. I stared at my notepad, pen in hand, while my mind raced to piece the clues together. Yet, something in my gut told me this wasn’t right. It was almost as if… someone was watching us from the shadows.
Self Hint:
User: (What's wrong here?)
Model:
<cot>I scan the area carefully, trying to locate those hidden eyes. There's no obvious cover around the factory, and no suspicious figures beyond the police tape.</cot>I frown, unconsciously touching the holster at my waist<cot>Is it just the wind? Or am I imagining things?</cot>I take a deep breath, trying to calm myself, but my heart keeps racing<cot>I need to find those eyes, I can't let anyone get close to this place.</cot>
Exploration Action:
User: (I look around)
Model:
I slowly turn my body, my gaze sweeping across every inch of the abandoned factory, from the corners to the windows, and out to the police tape<cot>There are a few pedestrians beyond the tape, looking calm as if nothing unusual is happening here. Something seems to move in the cobwebs in the corner, but when I look closer, there's nothing there.</cot>My eyes linger on a weathered wooden window, worn down by wind and rain<cot>Could that be someone's hiding spot?</cot>
### How to Use This LoRA:
1. Install [ms-swift](https://github.com/modelscope/ms-swift/tree/main/swift):
```bash
pip install ms-swift[llm] -U
- Use Swift for deploy:
RAY_memory_monitor_refresh_ms=0 CUDA_VISIBLE_DEVICES=0 swift deploy\
--model_type qwen2_5\
--model qwen/Qwen2.5-7B-Instruct\
--adapters mirau=mirau-7b-RP-base\ # the Lora you download
--infer_backend vllm --max_batch_size 1\
--max_length 8192 \
--max_model_len 8192 \
--port 8886 \
--host 0.0.0.0\
--vllm_max_lora_rank 128\
--merge_lora true # This merges the LoRA into the model, resulting in approximately 14GB of weights
