Core AI is Apple's on-device ML runtime in iOS 27 / macOS 27 and the successor to Core ML: PyTorch models are exported with Apple's coreai-torch (LLMs: coreai.llm.export) into .aimodel bundles that run on the GPU or the Neural Engine, e.g. Qwen3-8B 4-bit decodes at 94 tok/s on an M4 Max GPU, MLX 90 under the same protocol (apple-silicon-llm-bench, macOS 27 beta 26A5353q, 2026-06-11).

This model has no row on DeviceMark, the on-device LLM leaderboard.

Gemma 4 E4B (text) — Apple Core AI (.aimodel)

Gemma 4 E4B's text decoder converted to Apple's Core AI (the Core ML successor announced at WWDC26), running on iOS 27 / macOS 27 via Apple's coreai-pipelined GPU engine — zero custom kernels, greedy oracle 8/8 exact vs the fp32 Hugging Face reference on the Mac GPU and the iPhone GPU (iPhone is 24/24 token-identical to the Mac on the determinism probe).

Converted directly from Google's official QAT release google/gemma-4-E4B-it-qat-q4_0-unquantized: bf16 weights trained for q4_0 rounding, and q4_0 is this bundle's quantization class (per-block-32 absmax linear int4) — Google publishes these checkpoints as "preserving similar quality to bfloat16", so this int4 conversion carries that guarantee by design, not by post-hoc gating.

Requires iOS 27 / macOS 27. Conversion code, knowledge base, engine patch stack: coreai-model-zoo — model card: zoo/gemma4-e4b.md.

Use it

New to Core AI? Start with CoreAIKit 0.7.3. Follow its requirements and first-run steps for qwen3-0.6b, then open the same release's ChatDemo. The README records the tested OS/SDK and download size; model and device coverage is stated per example.

⚡ One line — run the kit's task op on this model (import CoreAIOps; no session, no model plumbing, downloads on first use):

let tldr = try await CoreAI.summarize(text, options: .model("gemma-4-e4b"))

Every op, one shape — Cookbook.

▶️ Run it (source) — the ChatDemo runner (GUI + CLI, one app for every chat model in the catalog):

git clone --branch 0.7.3 --depth 1 https://github.com/john-rocky/coreai-kit
export DEVELOPER_DIR=/Applications/Xcode-27.0.0-RC.app/Contents/Developer
open -a /Applications/Xcode-27.0.0-RC.app coreai-kit/Examples/ChatDemo/ChatDemo.xcodeproj
# → Run, then pick "Gemma 4 E4B" in the model picker

# agents / headless (macOS):
cd coreai-kit/Examples/ChatDemo
swift run -c release chat-cli --model gemma-4-e4b --prompt "What can you do, offline?"

Use Xcode build 27A266a from the release's .xcode-pin; adjust the app path if your installation is named differently.

💻 Build with it — complete; the glue is kit API, copy-paste runs:

import CoreAIKit

let id = "gemma-4-e4b"
let chat: ChatSession
if id == "qwen3-0.6b" {
    // Freeze the release starter; other selections retain the live catalog's
    // model-specific dispatch (including paired Gemma bundles).
    guard let model = ModelCatalog.builtin.entry(id: id)?.modelID else {
        throw CoreAIKitError.modelNotAvailableOnPlatform(id: id)
    }
    chat = try await ChatSession(model: model)
} else {
    chat = try await ChatSession(catalog: "gemma-4-e4b")
}
let reply = try await chat.respond(to: prompt)
// reply: the answer, generated fully on-device

When Apple's FoundationModels built-in model isn't enough, keep your session code and swap the model — one line. CoreAIKit's KitGemmaModel plugs supported chat bundles into the system LanguageModelSession. Tool calling depends on the model's dialect; guided generation also requires a compatible sequential engine. Check the linked support matrix before using either capability.

The take-home is Examples/ChatDemo/Sources/QuickStart.swift — this exact code as one typed function, no UI; the CLI is an argument shell over it, and the GUI drives the same ChatSession across turns for its transcript. Multi-turn? Hold the ChatSession and call respond(to:) per turn — it keeps the conversation history; streamResponse(to:) yields tokens as they decode.

Integration checklist

• SPM: https://github.com/john-rocky/coreai-kit (exact 0.7.3) → product CoreAIKit
• Info.plist: none needed
• Entitlements: none needed (macOS)
• First run downloads the model — ~7,590 MB (Mac) — then it loads from the local cache (Application Support; progress via the downloadProgress callback)
• Measure in Release — Debug is ~3× slower on per-token host work

Measured (greedy; M4 Max / iPhone 17 Pro, settled device)

On iPhone the working set stays tiny — measured peak footprint 2.2 GB (4.2 GB headroom): the PLE table rides as a clean mmap and the AOT executable pages are evictable. Both phases land exactly on the bandwidth model (~2.1 GB int4/token).

What E4B is (config + checkpoint verified)

Clean dense model — no MoE. 42 layers (full attention every 6th), hidden 2560, intermediate 10240 uniform, 8 query heads / 2 KV heads, dual head_dim 256/512, 18 KV-shared layers (the engine bundle stacks the 24 non-shared layers into ONE unified padded KV pair), per-layer embeddings (the [262144, 10752] int8 table ships in ios-frontend/gemma4_e4b_qat_gather_raw/), final-logit softcap 30. The QAT checkpoint prunes the never-used KV projections on the shared layers — the zoo's loader handles both layouts.

Run contract (each item is load-bearing)

Full story + traps: pipelined-engine page.

1. Swift stack = apple/coreai-models + the zoo's patch stack (apps/*.patch, in order). The ★ provider bundle needs EngineOptions.perTokenInputProvider (coreai-pipelined-per-token-inputs.patch); the tbl bundle needs EngineOptions.staticInputBuffers (coreai-pipelined-static-inputs.patch).
2. Provider mode: per token, fill ple_tokens [1,1,42,256] fp16 from the table dump — row = i8[id] * scale[id] * sqrt(256), mmap-gathered (~0.1 ms). tbl mode: bind ple_table ← embed_per_layer.i8 and ple_scale ← embed_per_layer.scale.f32 as OWNED storageModeShared MTLBuffers (buffer-backing traps in the knowledge page).
3. COREAI_CHUNK_THRESHOLD=1 before engine creation; never call engine.warmup() (S=1 graph; a 1-token generate after load is the warmup).
4. iPhone: AOT is mandatory (the 3.7 GB-constants graph crashes the on-device specializer) — use the precompiled _aotc_h18p/ bundle, or xcrun coreai-build compile <bundle>.aimodel --platform iOS --preferred-compute gpu --architecture h18p --expect-frequent-reshapes and point metadata.json's assets.main at the .aimodelc. Ship the com.apple.developer.kernel.increased-memory-limit entitlement as headroom insurance, and bench a settled device (a just-unlocked iPhone under-reads ~35%).

Reproduce from scratch (oracle + tables are checkpoint-derived — regenerate for any new weights): conversion/export_gemma4_decode_pipelined.py with --hf-id google/gemma-4-E4B-it-qat-q4_0-unquantized.

License

Gemma 4 is released under the Apache License 2.0 — see the base card google/gemma-4-E4B-it-qat-q4_0-unquantized and Google's Gemma 4 license page. These .aimodel bundles redistribute the weights in a different serialization; the conversion adds no additional restrictions.

Sibling repo (E2B, incl. its own official-QAT bundles): gemma-4-E2B-CoreAI.

More models in this format: Core AI Model Zoo — 75 models, each with the recipe that produced it.

Want a different model on-device? Open a request — free, open weights only; the export and its measured numbers get published publicly.