pkm-cabt-ppo — self-play agents for the Pokémon TCG (cabt engine)
Policy/value networks trained by PPO self-play to play the Pokémon Trading Card
Game inside the kaggle_environments cabt engine.
Code: https://github.com/jibai-devs/pkm
Model
A pointer-style policy/value network. The engine hands the agent a variable-length list of legal options each decision, so the network cannot use a fixed action head:
- The board state is embedded into a vector
h. - Each option is embedded into
o_iand scored againsth(plus a summary of the options already picked this decision), giving a softmax over the variable-length option list. - Multi-select decisions decompose into sequential picks. A learned STOP row is
appended to the option list, becomes legal once
minCountpicks are made, and ends the sequence when chosen. A decision's log-prob is the sum of its picks'.
Sizes: card embedding 32, attack embedding 16, option-type embedding 8, option encoding 64, hidden 128. Roughly 0.5M parameters — small, because the hard part here is the action space, not capacity.
Files
| File | What it is |
|---|---|
policy.npz | numpy export of the 02_dragapult net — torch-free inference |
02_dragapult/ppo_latest.pt | PPO checkpoint, 10,000 iterations (default agent) |
01_psychic/ppo_latest.pt | PPO checkpoint, 1,000 iterations |
00_basic/ppo_latest.pt | PPO checkpoint, 200 iterations |
00_basic/exit_latest.pt | expert-iteration (IS-MCTS distillation) checkpoint |
*/deck.csv | the 60-card deck each agent was trained on |
*/ppo_train.csv | per-iteration training metrics |
policy.npz exists because the Kaggle submission bundle has a ~197 MiB cap and
torch does not fit. pkm/rl/numpy_policy.py replays the same forward pass in
numpy, so the shipped agent runs on numpy alone.
The weights are deck-specific. Card identity is embedded, and each agent
self-played only against its own deck — load a checkpoint with the wrong deck and
it will play badly. The matching deck.csv is in each directory.
Results
Evaluation win rate against a random-legal-move agent, at the final iteration:
| Agent | Iterations | Eval win rate | Games |
|---|---|---|---|
02_dragapult | 10,000 | 100% | 20 |
01_psychic | 1,000 | 95% | 20 |
00_basic | 200 | ~80% (as documented in-repo) | — |
These are self-play agents measured against random, on a 20-game eval, against their own deck. That is a low bar and a small sample: read them as "training is working," not as a strength rating. No head-to-head vs. IS-MCTS or vs. human play is reported here.
Usage
from huggingface_hub import hf_hub_download
npz = hf_hub_download("TomatoCream/pkm-cabt-ppo", "policy.npz")
ckpt = hf_hub_download("TomatoCream/pkm-cabt-ppo", "02_dragapult/ppo_latest.pt")
deck = hf_hub_download("TomatoCream/pkm-cabt-ppo", "02_dragapult/deck.csv")
Load the torch checkpoint with the network from the repo:
import torch
from pkm.rl.model import PolicyValueNet
model = PolicyValueNet()
model.load_state_dict(torch.load(ckpt, map_location="cpu", weights_only=True))
model.eval()
Or run torch-free, the way the Kaggle submission does, via
pkm/rl/numpy_policy.py against policy.npz.
Training
PPO self-play with checkpoint-pool opponent sampling and potential-based reward
shaping on the prize differential. 00_basic additionally has an
expert-iteration checkpoint: IS-MCTS with determinization (for imperfect
information) generates targets, which are distilled back into the network.
just train 200 16 deck/01_psychic.csv # PPO self-play
just exit-train # expert iteration
just export # -> policy.npz
Limitations
- Deck-specific, as above; no multi-deck or opponent-pool generalization yet.
- Evaluated only against a random agent on a small sample.
- Tied to the
cabtengine's observation schema; the encoder assumes it.