PPO Agent for CartPole-v1

This repository contains a PPO Actor-Critic agent trained on the Gymnasium CartPole-v1 environment as part of Machine Learning Experiment 5.

Experiment configuration

• Algorithm: Proximal Policy Optimization (PPO)
• Environment: CartPole-v1
• Framework: PyTorch
• Random seed: 42
• Clip coefficient: 0.2
• Training steps: 19968
• Parallel environments: 4
• Rollout steps: 128
• Learning rate: 0.00025

Evaluation results

• Evaluation episodes: 20
• Mean reward: 442.50
• Standard deviation: 75.60
• Minimum reward: 299.00
• Maximum reward: 500.00

Debugging modifications

The original tutorial used an outdated Gym interface. The implementation was updated to Gymnasium and modified to support the current reset() and step() return formats. The Actor-Critic network, PPO training loop, evaluation procedure, model saving, training visualization, and video recording pipeline were successfully executed on Google Colab.

Repository contents

• ppo_cartpole_task1_seed42.pt: trained PyTorch checkpoint
• results.json: quantitative evaluation results
• task1_cartpole_training_curve.png: training reward curve
• ppo_cartpole_demo.mp4: evaluation video