Reinforcement learning on Upkie

A project log for Upkie wheeled biped robots

Wheeled biped robots that can balance, go around and more. Upkies are fully open source, both hardware and software.

upkie-zeroUpkie Zero 12/09/2023 at 15:530 Comments

The latest release (v3.0.0) of Upkie's software brings a functional reinforcement learning pipeline with sim-to-real transfer. The pipeline is based on Stable Baselines3, with standard sim-to-real tricks that could very well work on other wheeled biped robots. The pipeline trains on the Gymnasium environments in upkie.envs (pip-installable from PyPI) and is implemented in the PPO balancer. Here is a policy trained in Bullet and running on a real Upkie:

There is also a usage video showing how to run the pipeline:

Hoping this helps newcomers get started with reinforcement learning on real robots!