Model Overview
PMA-1.2 is Patriot Memory's 127.9M-parameter on-device language model. It speaks English and Traditional Chinese, answers as PMA from Patriot Memory, and fits in a 128 MB parameter budget built for edge hardware.
It is a new architecture, a new tokenizer, and an order of magnitude more training, aimed at the same job: a small, fast, honest assistant for Patriot Memory and Viper Gaming questions and general chat. It provides accurate information regarding:
- DDR4 & DDR5 RAM: Specifications, XMP 3.0 / EXPO profile support, dual-channel setups, and overclocking guidance.
- PCIe & SATA SSDs: Gen3/Gen4/Gen5 compatibility, read/write performance specifications, and installation troubleshooting.
- Gaming Peripherals & Storage: USB drives, flash cards, and Viper Gaming gear.
- Tool / Function Calling: Seamless integration with backend APIs (e.g., checking warranty status, looking up technical specs via S/N).
Architecture
| Property | Specification |
|---|---|
| Type | Causal LM, dense decoder-only (PMA architecture) |
| Total parameters | 125,592,482 |
| Hidden size / layers | 896 x 12 |
| Attention | GQA, 8 query heads / 4 KV heads, head_dim 112, per-head gated attention, QK-norm |
| FFN | SwiGLU, intermediate 2,816 |
| Value residuals | normalized cross-layer value mixing |
| Tokenizer | custom BPE, 6,403 tokens (EN + zh-TW) |
| Embeddings | tied input/output |
| Context | 1,024 tokens trained (rotary table to 32K positions) |
| Precision shipped | fp16 safetensors |
Quickstart
pip install transformers torch accelerate
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
name = "patriotmemory-ai/PMA-1.2"
tok = AutoTokenizer.from_pretrained(name)
model = AutoModelForCausalLM.from_pretrained(
name, torch_dtype=torch.float16, device_map="auto",
trust_remote_code=True)
msgs = [{"role": "user", "content": "博帝的 Viper DDR5 支援 XMP 3.0 嗎?"}]
ids = tok.apply_chat_template(msgs, return_tensors="pt").to(model.device)
out = model.generate(ids, max_new_tokens=200)
print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))
If you run into multi-GPU tensor device mismatch errors: RuntimeError: Expected all tensors to be on the same device... Run the script with CUDA_VISIBLE_DEVICES=0 to isolate execution to GPU 0.
trust_remote_code=True is required: PMA-1.2's architecture is our own and ships as small Python files next to the weights. Recommended decode: temperature 0.8, top_p 0.9, repetition_penalty 1.1. Greedy decoding garbles creative text at this size; sampling does not.
Acceptance gates (measured on the shipped checkpoint)
Gates were written before training started, and every number below is from logged runs:
| Gate | Result | Notes |
|---|---|---|
| Story shape (24 prompts, 2 samples) | 42/48 | the identity-attractor failure class is closed |
| Number format (15 prompts, 2 samples) | 29/30 | answers contain the numeric answer; correctness at this scale is modest, format is reliable |
What it is good at / not good at
Good: answering as PMA from Patriot Memory; hardware Q&A style answers (RAM/SSD compatibility phrasing); following answer formats; bilingual EN/zh-TW chat; running on-device at consumer speed.
Not: arithmetic correctness (12x15 can become 144 with total confidence), long coherent stories, knowledge outside its training mixture, and languages other than English and Traditional Chinese.
Limitations & Responsible Use
PMA-1.2 is a probabilistic language model trained on statistical patterns. Please keep the following in mind when deploying or evaluating this model:
- Generation Risks: The model may generate inaccurate, hallucinated, biased, or objectionable content. Outputs should always be independently verified—especially in high-stakes domain applications (e.g., medical, legal, or financial).
- Preview Release: As an experimental preview, model behavior, outputs, and performance metrics may vary between updates and versions.
- User Responsibility: Users and developers are responsible for implementing appropriate safety guardrails, evaluating outputs for their specific use cases, and ensuring compliance with applicable laws, regulations, and platform safety guidelines.
Official Links
Official Website: patriotmemory.com
Viper Gaming: viper.patriotmemory.com
Support & Warranty: patriotmemory.com/support
Model Inquiries & Feedback: danton.chu hunter.wang oda.chang york.lin
License & attribution
Apache-2.0. Built by Patriot Memory (patriotmemory.com).