DeepAnalyze-8B (Q4_K_M GGUF)

DeepAnalyze-8B is a specialized, quantized large language model designed for autonomous in-memory data analytics. It translates natural language instructions into deterministic state-machine logic, executing complex operations on hierarchical spreadsheets, tabular datasets, and SQL databases.

This GGUF distribution is the official backend engine for the DeepAnalyze IPython Extension, an agentic execution harness that handles AST linting, runtime self-repair loops, and zero-copy DuckDB SQL querying directly inside Jupyter/IPython sessions.

Model Details

• Base Architecture: DeepSeek-Coder (8 Billion Parameters)
• Training Corpus: 500,000+ instruction-response pairs focusing on advanced Pandas operations, DuckDB, and code-to-chart synthesis.
• Quantization Format: Q4_K_M (4-bit Medium) via llama.cpp.
• Context Length: 8,192 tokens natively.
• Hardware Requirements: Extremely lightweight footprint requiring ~5.5 GB of RAM/VRAM for inference.

Why Q4_K_M?

The 4-bit medium quantization strikes the optimal balance between high-speed token generation and strict syntax adherence. Because the model operates within a closed-loop AST self-repair environment (via the IPython extension), the engine automatically catches and corrects the marginal syntax degradation introduced by 4-bit quantization, yielding maximum performance with minimal hardware overhead.

DeepAnalyze-8b: Agentic In-Memory Data Execution Engine

DeepAnalyze-8b is an open-weights, 8-billion parameter code and reasoning model fine-tuned for high-speed, local-only data science and tabular analytics.

About This Project & Creator's Note

Hi! I’m a recent graduate who built this project as a hands-on learning lab to dive deep into large language models and solve real-world data analysis headaches.

When doing data analysis, I found myself spending hours writing repetitive data-cleaning scripts, dealing with messy ERP invoice reports, and looking up syntax rather than focusing on the actual data insights. Instead of just using existing AI chat tools, I wanted to understand what makes these systems work under the hood—how to fine-tune a model, structure GGUF weights, handle token triggers, and build a local agent harness from scratch.

This project is an ongoing personal experiment built out of curiosity and the joy of tinkering. It’s tailored specifically around my own daily workflow as an aspiring data analyst—giving me a private, blazing-fast local co-pilot that helps me learn, explore data faster, and practice modern software and data engineering principles.

Intended Ecosystem & Architecture

While the model weights can be loaded into standard chat interfaces (like LM Studio or text-generation-webui), DeepAnalyze is not designed for generic conversational chat.

DeepAnalyze is explicitly trained to output executable code payloads encased in specialized <Execute> tags. To utilize the model as intended, it must be paired with its agentic execution harness in IPython/Jupyter.

Primary Capabilities (via the Agent Harness)

• Bilingual DataFrame Adapter: Native polymorphic execution across both Polars (multi-threaded Rust engine) and Pandas.
• Zero-Copy In-Memory SQL: Microsecond analytical queries powered by DuckDB and Apache Arrow shared memory buffers.
• Hierarchical Unravelling: Autonomous unravelling of multi-row, non-rectangular ERP, SAP, and accounting exports.
• AST Security Sandboxing: Zero-trust Abstract Syntax Tree auditing that parses and blocks unauthorized system calls (os, subprocess, socket, requests) prior to execution.
• State Rollback Engine: Non-destructive session snapshotting with instant rollback (--undo).
• Automated Feature Engineering: Rapid vector transformations, statistical profiling, and code-to-chart visualization synthesis.

👉 View the full system architecture and setup guide on GitHub

Model Serving (llama.cpp)

Initialize the model locally as an OpenAI-compatible API server using llama-server. This exposes port 8080 for the IPython agent to connect to:

llama-server \
  --hf-repo aboOod3d/deepanalyze-8b \
  --hf-file deepanalyze-8b.gguf \
  --port 8080 \
  -c 8192 \
  -ngl 99 \
  --flash-attn

Quickstart & Jupyter Integration

1. Install the Extension

Install the execution package directly from GitHub into your Python environment:

pip install git+[https://github.com/abdullah-binmadhi/Deepanalyzer-8b.git](https://github.com/abdullah-binmadhi/Deepanalyzer-8b.git)

2. Load in Jupyter / IPython

Load the magic extension and run queries with automated tab-completion and Rich telemetry enabled:

%load_ext deepanalyze

# Execute SQL query over existing in-memory DataFrames using Polars & DuckDB
%deepanalyze -s "Calculate total revenue by customer segment and filter for completed orders"

Prompt Engineering & Chat Template (For API & Direct Usage)

Note: If you are using the official %deepanalyze IPython extension, all prompt scaffolding, token triggers, and execution extraction are handled automatically.

If you are calling the model directly via the llama-server OpenAI endpoint or building a custom client or autonomous agent, you must follow the prompt contract below.

1. The Analytic Trigger

The prompt string must terminate with the <Analyze> token. This acts as a cognitive trigger, forcing the model out of conversational mode and into strict reasoning and code-generation mode:

<｜User｜> {Your data cleaning or transformation instructions} <｜Assistant｜><Analyze>

2. The Execution Sandbox Block

The model emits executable Python/SQL state-machine code strictly within <Execute> delimiters. The host application is expected to parse these tags, validate the Abstract Syntax Tree (AST), and execute the payload against active in-memory data:

I have evaluated the schema and data types. Here is the restructuring logic:

<Execute>
import polars as pl
import duckdb

# Zero-copy query directly against local in-memory DataFrames
query = """
    SELECT 
        category, 
        SUM(revenue) AS total_revenue,
        COUNT(*) AS transaction_count
    FROM df 
    WHERE status = 'Paid'
    GROUP BY 1
    ORDER BY total_revenue DESC
"""
cleaned_df = duckdb.query(query).pl()
</Execute>

Intended Use & Security

• Target Hardware: Optimized for Apple Silicon (M-series unified memory) and standard local CUDA environments.
• Privacy by Design: Raw data values never leave local memory. The engine supports structural geometry masking and reversible surrogate tokenization for enterprise compliance.
• Primary Domains: Exploratory Data Analysis (EDA), ERP/accounting restructuring, automated data cleaning, and statistical profiling.

Citation & Attribution

This project builds upon the foundational research and dataset provided by RUC-DataLab.

• Base Weights & Research: RUC-DataLab/DeepAnalyze-8B
• Dataset: RUC-DataLab/DataScience-Instruct-500K
• License: MIT License