MrBalance
Mr. Balance is a plate, and his only goal is to balance whatever object is placed onto him.
Mr. Balance hates when objects fall, so he tries to keep them balanced.
Model Architecture
- Base Architecture:
MrBalanceMLPForRL - Observation Size:
64 - Hidden Size:
128(first, second and third layer) - Bottleneck Size:
64(last layer) - Layers:
4 - Action Space:
2
Mr. Balance uses a shared Actor-Critic MLP designed for continuous-control reinforcement learning. The 64-dimensional observation vector is passed through the four layers with widths of 128 for the first 3 layers and 64 for the last one, using SiLU activations between each of them. The resulting 64-dimensional bottleneck representation is shared by both the actor and critic. The actor produces two control outputs corresponding to the plate's roll and pitch axes, using a Gaussian policy with learned action standard deviations and Tanh squashing to constrain actions to [-1, 1]. The critic uses the same representation to estimate the scalar state value V(s). This shared architecture keeps the policy net small while allowing it to combine object position, velocity, orientation, plate state, and object-property stuff into a combined control representation.
Training Configuration
- Number of Environments:
8 - Rollout Steps:
512 - PPO Epochs:
6 - Minibatch Size:
2048 - Gamma:
0.99 - GAE Lambda:
0.95 - Clip Coefficient:
0.20 - Value Coefficient:
0.50 - Entropy Coefficient:
0.005 - Maximum Gradient Norm:
0.50 - Learning Rate:
3e-4 - Adam Epsilon:
1e-5 - Anneal Learning Rate:
true - Number of Episodes:
12756 - Maximum Episode Steps:
10000
Training Objects
- Sphere
- Egg
- Heavy Ball
Training Results
| Object | Reward Mean | Len Mean | Survival % | Tracking Err |
|---|---|---|---|---|
| sphere | 24,398.48 | 10,000.0 | 100.0% | 0.0205m |
| egg | 23,240.42 | 10,000.0 | 100.0% | 0.0458m |
| heavy_ball | 24,310.68 | 10,000.0 | 100.0% | 0.0401m |
Mr. Balance excels at balancing all the objects he was trained on, but does he generalize to other objects?
| Object | Reward Mean | Len Mean | Survival % | Tracking Err |
|---|---|---|---|---|
| sphere | 24,398.48 | 10,000.0 | 100.0% | 0.0205m |
| disk | 21,191.56 | 10,000.0 | 100.0% | 0.1362m |
| egg | 23,240.42 | 10,000.0 | 100.0% | 0.0458m |
| cup | 21,702.17 | 10,000.0 | 100.0% | 0.1242m |
| coin | 20,973.76 | 10,000.0 | 100.0% | 0.1428m |
| stick | 13,601.42 | 8,477.4 | 80.0% | 0.3399m |
| tall | 22,265.77 | 10,000.0 | 100.0% | 0.1077m |
| triangle | 18,794.99 | 10,000.0 | 100.0% | 0.2181m |
| block | 21,489.38 | 10,000.0 | 100.0% | 0.1316m |
| puck | 21,265.33 | 10,000.0 | 100.0% | 0.1361m |
| cone | 18,327.38 | 10,000.0 | 100.0% | 0.2316m |
| capsule | 22,302.92 | 10,000.0 | 100.0% | 0.1097m |
| wedge | 18,921.33 | 10,000.0 | 100.0% | 0.2148m |
| tetra | 18,838.96 | 10,000.0 | 100.0% | 0.2165m |
| flat_bar | 20,550.93 | 10,000.0 | 100.0% | 0.1526m |
| cross | 21,394.63 | 10,000.0 | 100.0% | 0.1435m |
| L_shape | 21,221.41 | 10,000.0 | 100.0% | 0.1389m |
| wide_block | 20,857.84 | 10,000.0 | 100.0% | 0.1611m |
| heavy_ball | 24,310.68 | 10,000.0 | 100.0% | 0.0401m |
| offcenter_block | 21,408.22 | 10,000.0 | 100.0% | 0.1395m |
The answer is yes; Mr. Balance excels at balancing a wide range of objects, not just the three he was trained on.
Inference
First install the required dependencies:
pip install torch transformers safetensors "mujoco==3.10.0" numpy
Windows users also might need the Microsoft C++ Build Tools:
https://visualstudio.microsoft.com/visual-cpp-build-tools/
Next, download both inference.py and balance_plate_rl.py and run:
python inference.py
Add --render for live visual rendering or/and --object to choose a specific object.
License
Apache 2.0.
Citation
@misc{mrbalance,
title = {Mr. Balance: Teaching RL agents to Balance Objects},
organization = {FromZero},
authors = {Paul Courneya},
year = {2026},
url = {https://huggingface.co/fromziro/MrBalance]
}