PolyClerk-12B
PolyClerk-12B (12B active / 106B total MoE parameters, GLM-4.5-Air derivative) is fine-tuned for legal work product: counterparty markup analysis, redlining, contract drafting, and clause-level document review over very long contexts (up to 128k tokens).
Lineage
- Base: zai-org/GLM-4.5-Air (MIT)
- Stage 1: polygramme/Rolodex-12B — OAPL training on agentic tool-use trajectories
- Stage 2 (this model): LoRA fine-tune (r=32, α=64) on legal-bench work-product tasks, merged into the stage-1 weights. This repo contains the fully merged weights — no adapter loading required.
Training
| Method | OAPL, LoRA r=32 / α=64 (merged) |
| Framework | ms-swift (Megatron backend), TP4 × CP2 |
| Hardware | 8× H200 |
| Sequence length | 131,072 |
| Epochs | 1 |
Evaluation
On a held-out legal work-product benchmark (whole-document mode, long-context markup/review tasks), this model scores comparably to frontier closed models on the small evaluated task set. Numbers are from a limited sample (N=3 task families) — treat as indicative, not definitive.
⚠️ Contamination note: this model was trained on tasks drawn from the LAB legal benchmark family. Do not use LAB (or derivative benchmarks) to evaluate this model.
Usage
Requires ~200GB of weights (bf16). Serve with vLLM:
vllm serve polygramme/PolyClerk-12B --tensor-parallel-size 4 --max-model-len 131072
Or load with transformers (multi-GPU required):
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("polygramme/PolyClerk-12B")
model = AutoModelForCausalLM.from_pretrained("polygramme/PolyClerk-12B", device_map="auto", torch_dtype="bfloat16")
The chat template is included (chat_template.jinja).
Intended use & limitations
Intended for legal document analysis workflows (markup review, redline drafting, provision-level analysis). Outputs are not legal advice; a qualified lawyer must review all work product. The model may hallucinate section references or values on documents unlike its training distribution — verify against source documents.
Training data attribution
Fine-tuned on tasks from harvey-labs (MIT License, © 2026 Harvey AI). The MIT permission notice is reproduced here in accordance with the license:
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction [...] subject to inclusion of the above copyright notice and this permission notice in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND.
License
MIT, following the GLM-4.5-Air base license. © the model authors.