laguna-s-2.1-Q4_K_M-layers

GGUF layer package for running laguna-s-2.1-Q4_K_M across a local Mesh LLM cluster.

This package is derived from poolside/Laguna-S-2.1-GGUF and keeps the original GGUF distribution split into per-layer artifacts for distributed inference.

Highlights

Model Overview

Recommended Use

• Local and private inference with Mesh LLM.
• Multi-machine serving when the full GGUF is too large for one host.
• OpenAI-compatible chat/completions workflows through Mesh LLM's local API.

For upstream architecture details, chat template guidance, sampling recommendations, license terms, and benchmark notes, see the source model card: poolside/Laguna-S-2.1-GGUF.

Default Serving Policy

This pinned Q4_K_M package selects request-local suffix N-gram speculation by default:

• suffix match length: 5 to 32 tokens
• maximum proposal: 48 tokens
• verification window: 1 to 32 tokens
• verification pipeline depth: 2

No speculative CLI flags are required on a Mesh LLM build that supports the package window_policy.pipeline_depth field. Older builds ignore that optional field and retain their legacy verification depth of 1.

The default is based on an ordinary two-node Mesh run using an Apple M5 Max / Metal coordinator and an Australian Vast RTX 6000 Ada / CUDA worker. Normal placement selected layers 0..36 / 36..48, with F16 activation wire and a direct Iroh/QUIC path. A 44,460-token repeat/copy prompt plus 65 generated tokens produced identical content across three requests. Suffix N-gram accepted 32 of 64 proposed tokens in each request, reached pipeline depth 2 with two verification windows in flight, and decoded at 19.74, 18.24, and 15.42 tok/s. The 63-request stability harness completed without an HTTP failure; the full retained capture contains 65/65 HTTP 200 responses.

This evidence does not certify structured tool calls, full 256K context saturation, Q8 activation wire, Q8 KV quality, DFlash, or any other Poolside model or quantization. The suffix predictor is workload-dependent: a low-overlap control produced no proposal.

Quickstart

# Run this on each machine that should contribute memory/compute.
mesh-llm serve --model "meshllm/laguna-s-2.1-Q4_K_M-layers" --split

# Check the mesh and discover the OpenAI-compatible model name.
curl -s http://localhost:3131/api/status
curl -s http://localhost:3131/v1/models

# Send an OpenAI-compatible chat request.
curl -s http://localhost:3131/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "poolside/Laguna-S-2.1-GGUF:Q4_K_M",
    "messages": [{"role": "user", "content": "Write a tiny hello-world function in Rust."}],
    "max_tokens": 128
  }'

Package Variant

What Is Included

Validation

Generated by the Mesh LLM HF Jobs splitter from mesh-llm ref feat/laguna-certification. Each artifact is checksummed as it is written, uploaded to this repository, and removed from the job workspace before the next artifact is produced.

skippy-model-package write-package "/source/laguna-s-2.1-Q4_K_M.gguf" --out-dir "/tmp/meshllm-layer-job-meshllm_laguna-s-2.1-Q4_K_M-layers-199/package"

Links

• Source model: poolside/Laguna-S-2.1-GGUF
• Mesh LLM website: meshllm.cloud
• Mesh LLM: github.com/Mesh-LLM/mesh-llm
• Discord: discord.gg/rs6fmc63eN
• Package catalog: meshllm/catalog
• Package format: layer-package-repos.md