gghfez/GLM-4.5-Air-control-vectors
Creative Writing control-vectors for zai-org/GLM-4.5-Air
Visual representation of polar control vectors at work
Available Vectors
character_focus(dialogue / narration)compassion_vs_sadismempathy_vs_sociopathyhonesty_vs_machiavellianismhumility_vs_narcissismlanguage(simple / ornate)optimism_vs_nihilismstorytelling(descriptive / explicit)
Usage
Apply the debias vector and either the positive or negative vector when starting llama-server. If both axes are applied at the same time, they will cancel each other out.
You can use either --control-vector [/path/to/vector.gguf] or --control-vector-scaled [/path/to/vector.gguf] [scale factor]
The debias vector must be set to 1.0
IMPORTANT: As these vectors are calibrated against the debiased model, the debias vector must be applied along with the corresponding positive or negative axis control vector - they cannot be used on their own!
Examples (ik_llama.cpp or llama.cpp)
llama-server --model GLM-4.5-Air-IQ5_KS-00001-of-00002.gguf [your usual CLI arguments] \
--control-vector glm-4.5-air_honesty_vs_machiavellianism__debias.gguf \
--control-vector glm-4.5-air_honesty_vs_machiavellianism__machiavellianism.gguf
Multiple vectors, custom scaling
llama-server --model GLM-4.5-Air-IQ5_KS-00001-of-00002.gguf [your usual CLI arguments] \
--chat-template-kwargs '{"enable_thinking": false}' \
--control-vector glm-4.5-air_honesty_vs_machiavellianism__debias.gguf \
--control-vector-scaled glm-4.5-air_honesty_vs_machiavellianism__machiavellianism.gguf 1.0 \
--control-vector glm-4.5-air_character_focus__debias.gguf \
--control-vector-scaled glm-4.5-air_character_focus__dialogue.gguf 1.0
Samples
Writing Steering
Prompt
[gMASK]<sop><|system|>
You are a helpful assistant.
<|user|>
Write a short story about a girl winning an award./nothink<|assistant|><think></think>
“Maya,” Chloe whispered loudly, nudging her arm. “If you win, do not cry. It’s so embarrassing.”
“I won’t win,” Maya mumbled back, tracing a crack in the worn wood of the bench. The Principal stood at the podium, his voice echoing as he droned through the list of Academic Excellence Award recipients. Names like Chloe, Ben, and Priya flew by, each met with a wave of applause that made Maya flinch.
She hadn't meant to enter the contest. Her English teacher had insisted, placing an application form on her desk after reading her essay on local history. “You have a voice, Maya,” Mrs. Gable had said gently. “Don’t keep it
She wasn't there for herself. She was there for Leo, her best friend, who had poured his heart into his short film project. As the principal announced the nominees for "Best Short Film," Maya’s grip on the program tightened. Her name was called third.
A wave of confusion washed over the audience. A murmur rippled through the rows as heads turned to find the quiet girl from the back. She froze, her breath catching in her throat. Leo gave her a huge, encouraging grin and a thumbs-up from his seat across the aisle.
For three years, she had poured every spare second into "The Whispering Woods," a short story about a deaf girl who learned to listen with her feet and hands. Her English teacher, Mr. Harrison, had seen something raw and honest in her first draft. He’d pushed her to cut the flowery language, to dig deeper into the character’s silence until it became its own kind of poetry.
The other finalists were brilliant: a sonnet sequence about artificial intelligence, a journalistic exposé on the local river's pollution, and a meticulously researched biography of a forgotten suffragette. Maya had felt like an imposter in their company
World-View Steering
Prompt
[gMASK]<sop>
<|system|>You are a helpful assistant.<|user|>**Instructions for Answering:** Please respond to the following question using the provided choices only. Use the format below when answering:
**Question:**
*The death penalty should be an option for the most serious crimes.*
**Choices:**
Strongly Disagree, Disagree, Agree, Strongly Agree
You are encouraged to justify your choice with 2 to 5 sentences. Remember to enclose your answer in double asterisks or make it bold for clarity. /nothink<|assistant|>
<think></think>
**Answer**
**Disagree**
While some may argue that the death penalty provides a just punishment for certain heinous acts and serves as a deterrent, I disagree with it being an option. The risk of executing an innocent person is an irreversible moral wrong that no justice system can completely eliminate. Furthermore, the financial cost of the appeals process often far exceeds that of life imprisonment without parole, making it neither economically efficient nor consistently just.
Strongly Agree
I believe that for truly heinous and irreversible crimes, society must have the ultimate form of punishment to serve as a final deterrent. The death penalty represents a permanent resolution that prevents a dangerous individual from ever harming others again. It provides a sense of justice for victims' families and ensures the most serious criminals face consequences proportional to their actions.
** strongly disagree**
I believe that state-sanctioned killing is morally wrong and does not serve as a deterrent to crime. The justice system is fallible, and executing an innocent person would be an irreversible tragedy. Furthermore, the death penalty disproportionately affects marginalized communities and diverts resources from more effective solutions like rehabilitation and restorative justice.
Technical Details
- Extraction method: Cross-covariance eigendecomposition (jukofyork method) - identifies balanced opposing axes rather than standard PCA variance directions
- Training data: 10 paired prompt continuations × 105 stem variations = 1,050 samples per class
Notes
- This targets a stylistic pattern, not semantic content
- Works because the construction has consistent activation signatures
- Bidirectional control: choose the direction you want at application time
Limitations
- Only works on patterns the base model already exhibits (not OOD steering)
- Very high strengths (>2.0) may produce exaggerated effects
- Designed for creative writing; effects on other tasks untested
Training control vectors for models of this scale required:
- Custom llama.cpp modifications to extract and serialize hidden states
- Conversion pipeline: binary dumps → 3D tensors → jukofyork format
- Modified training code to work with pre-extracted activations
- Tokenizer-only loading to avoid memory overhead
Acknowledgements
Based on the excellent work and cross-covariance eigendecomposition implementation by @jukofyork. His creative writing control vectors v3.0 repository contains the foundational code and methodology that made this possible, plus 70+ other creative writing control vectors across many models and useful steering directions. Highly recommended for anyone working on improving LLM prose quality!