Mr. Pong
In the realm of Ping Pong, Mr. Pong is no ordinary paddle. Feared across the table as the Blue Beast, he commands every rally with ruthless precision, striking despair into the hearts of all who dare face him.
In other words, he is a reinforcement learning (RL) agent trained to compete at a grandmaster level in Ping Pong.
Model Architecture
- Base Architecture:
MrPongMLPForRL - Hidden Size:
160 - Observation Size:
16 - Number Of Layers:
3 - Action Space:
3(stay, up, down) - Activation:
tanh
Mr. Pong uses a shared Actor-Critic MLP designed for 2D table tennis control. The 16-dimensional observation vector is passed through two hidden layers of 192 units each, with Tanh activations between them. The resulting 192-dimensional representation is shared by both the actor and critic heads. The actor outputs categorical logits across the 3 movement actions (stay, up, down), while the critic uses the same representation to estimate the scalar state value V(s). This shared setup keeps the network small and fast on CPU while letting the model combine ball coordinates, velocities, paddle momentum, raycasted intercept points, and opponent court openings into a single control decision.
Training Configuration
- Number of Environments:
12 - Rollout Steps: `128
- PPO Epochs:
4 - Minibatch Size:
64 - Gamma:
0.99 - GAE Lambda:
0.95 - Clip Coefficient:
0.20 - Value Coefficient:
0.50 - Entropy Coefficient:
0.02 - Maximum Gradient Norm:
0.75 - Learning Rate:
3.5e-4 - Adam Epsilon:
1e-5 - Anneal Learning Rate:
true - Total Timesteps:
10,000,000 - Maximum Rally Steps:
1500
Training Opponents
- Realistic Hard Logic (human-like perception horizon and bounce raycasting)
- Medium Logic (linear trajectory extrapolation)
- Minimax Lookahead (depth 1 and depth 2)
- Current Self-Play (mirror matches against the active policy)
- Historical Self-Play (past checkpoints sampled between 5 and 75 saves ago)
- Easy Logic (reaction delay with targeting noise)
- Impossible Hard Logic (0ms reaction time)
- Random Agent
Training Results
| Opponent Name | Win % | Draw % | Loss % | Record (W / D / L) | Avg Rally |
|---|---|---|---|---|---|
| Easy Logic | 99.7% | 0.0% | 0.3% | 997W / 0D / 3L | 1.7 hits |
| Medium Logic | 99.0% | 0.2% | 0.8% | 990W / 2D / 8L | 17.3 hits |
| Realistic Hard | 86.6% | 8.1% | 5.3% | 866W / 81D / 53L | 33.9 hits |
| Impossible Hard | 0.0% | 98.4% | 1.6% | 0W / 984D / 16L | 64.8 hits |
| Minimax Depth 1 | 43.0% | 55.8% | 1.2% | 430W / 558D / 12L | 32.2 hits |
| Minimax Depth 2 | 78.4% | 20.3% | 1.3% | 784W / 203D / 13L | 23.7 hits |
| Random Agent | 100.0% | 0.0% | 0.0% | 1000W / 0D / 0L | 0.9 hits |
| Self-Play Mirror | 1.6% | 96.1% | 2.3% | 16W / 961D / 23L | 50.5 hits |
Mr. Pong has demonstrated the he has mastered the game Ping Pong; losing only <=5% of games.
Inference
First install the required dependencies:
pip install torch transformers
Next, download inference.py and run:
python inference.py
- Use
--mode playto play against the agent live in the terminal. - Use
--mode simulateto run AI match simulations (default). - Add
--opponentto choose a specific opponent (realistic_hard, medium, minimax_d1, minimax_d2, impossible_hard, easy, random). - Add
--videoto export an MP4 recording of the match.
License
Apache 2.0.
Citation
@misc{mrpong,
title = {Mr. Pong: Teaching RL agents to Play Ping Pong},
organization = {FromZero},
authors = {Paul Courneya},
year = {2026},
url = {https://huggingface.co/fromziro/MrPong]
}