Polymarket short-horizon direction signals
Small, honest models that score a prediction market's short-horizon price
direction from microstructure. Trained on real Polymarket data, validated
strictly out-of-time (temporal split + walk-forward), and benchmarked against
a distilled time-series foundation model. Methodology, sources and ablations:
see RESEARCH.md in this repo.
Models
seq_model.safetensors — the served model. An MLX feature-MLP / GRU over 13
close-price features (incl. proper Wilder RSI, CCI, MACD-hist, stochastic %K).
Winner feature_mlp, out-of-time val AUC 0.6502; walk-forward
mean AUC 0.6281 (min 0.603) across 4 time folds.
ohlcv_model.safetensors — richer model on true OHLCV (open/high/low/close/
volume/trade_count) with 20 features (typical-price CCI, stochastic/Williams on
true high-low, ATR, Parkinson vol, order-flow imbalance). Head-to-head:
{"feature_mlp": 0.5993, "seq_gru": 0.5961, "gbdt": 0.6026, "ensemble": 0.6082} — a LightGBM GBDT beats the small MLP/GRU, and the 3-way
ensemble is best (matching published tabular-vs-deep evidence).
bigdata/bigdata_model.safetensors + bigdata_gbdt.txt — the flagship, trained on orderfilled tape (1.2B trades) → 300s OHLCV + true order-flow (20785510 windows from 2000 tokens, 21420590 hourly bars). Real order-flow imbalance from each trade's aggressor side. Out-of-time AUC {"gbdt": 0.5799, "neural": 0.5745, "ensemble": 0.5814}; walk-forward mean 0.5784. Top features: ['williams_r', 'vol', 'last', 'band_z', 'stoch_k', 'atr'].
H100 experiments
- Distillation (
metrics/distill_metrics.json): Chronos-Bolt-Base teacher → tiny student, with a no-KD ablation. KD gain 0.0027 (neutral) — Chronos zero-shot is near-chance on this task, so it can't transfer. - Multilingual (
metrics/multilingual_metrics.json): multilingual question embeddings; cross-lingual cosine to EN mean 0.892 (6 languages). Text-only AUC 0.6099 but redundant with price for short-horizon direction (adds -0.0017).
Honesty notes
Metrics are out-of-time, not on an inflated split. up_rate_spread (top vs
bottom quintile hit-rate) is the trade-relevant headline. Chronos/foundation
models are baselines here, not the primary signal. Not financial advice.
Inference: load the safetensors with the matching *_normalizer.json (feature
order + z-score stats). code/mcp_server.py serves the close-only model live
over the Model Context Protocol.