Gemma 4 E2B Uncensored Slim — Q4_K_M GGUF
A compact Q4_K_M requantized GGUF build of a Gemma-based "uncensored slim" derivative, prepared primarily for efficient on-device inference.
This release is focused on Android, mobile-class hardware and
llama.cpp compatible runtimes.
The model can run fully locally:
- no cloud inference
- no remote API
- no API key
- no mandatory internet connection after download
It is used experimentally with NLLM, an Android-oriented local LLM runtime built around llama.cpp.
Why Q4_K_M?
On mobile devices, memory footprint and memory bandwidth are often the main bottlenecks.
Q4_K_M reduces model weight precision to approximately 4-bit while retaining higher precision for selected tensors where additional accuracy is useful.
The goal of this requantized build is to provide a practical balance between:
- model size
- RAM usage
- inference speed
- response quality
This makes the model considerably easier to deploy on constrained hardware than a higher-precision checkpoint.
Model File
| Field | Value |
|---|---|
| File | gemma-4-E2B-uncensored-slim-Q4_K_M.gguf |
| Quantization | Q4_K_M |
| Size | ~2.72 GB |
| Format | GGUF |
| SHA256 | adce3282cb2e87c1f878a92f9e55d9c0ba9f40fda7367dc535aabff13b6be31b |
Running with llama.cpp
CLI
llama-cli \
-m gemma-4-E2B-uncensored-slim-Q4_K_M.gguf \
-ngl 0 \
-c 2048 \
-p "Merhaba, nasilsin?"
Local OpenAI-compatible server
llama-server \
-m gemma-4-E2B-uncensored-slim-Q4_K_M.gguf \
-c 4096 \
--port 8080
Android / NLLM
Download the GGUF file to the device and select it as the active model inside NLLM.
The native inference layer uses an arm64-v8a build of llama.cpp.
Tested Hardware
| Field | Result |
|---|---|
| Device | TECNO KM8n (arm64-v8a) |
| Architecture | arm64-v8a |
| Approx. model RAM requirement | ~3 GB |
| Suggested system RAM | 4 GB minimum |
| Measured CPU inference | ~5 token/s |
Performance varies significantly depending on CPU architecture, memory bandwidth, thermal limits, context length and background load.
The measurement above refers to CPU inference without GPU acceleration.
Requantization Notes
This repository is primarily about deployment optimization, not language fine-tuning.
The model has been prepared as a smaller Q4_K_M GGUF build so it can be used more comfortably on mobile and local devices.
Turkish capability comes from the underlying model rather than from a dedicated Turkish fine-tuning stage.
Intended Use
Suitable for experimentation with:
- local assistants
- Android LLM applications
- offline inference
- lightweight agents
- mobile AI research
- Turkish / English conversational workloads
License
This model is Gemma-based.
Use is subject to the Gemma Terms of Use and Gemma Prohibited Use Policy.
Users are responsible for complying with the applicable Gemma terms.
English Summary
Q4_K_M requantization of a Gemma-based "uncensored slim" derivative, prepared for efficient fully on-device inference on Android and other resource-constrained systems through llama.cpp.
This is not a Turkish fine-tune. Its Turkish capability comes from the underlying model; this repository focuses on reducing deployment footprint and making local mobile inference practical.