Run Aloha (Real Robot)

This example demonstrates how to run with a real robot using an ALOHA setup.

Prerequisites

This repo uses a fork of the ALOHA repo, with very minor modifications to use Realsense cameras.

Follow the hardware installation instructions in the ALOHA repo.
Modify the third_party/aloha/aloha_scripts/realsense_publisher.py file to use serial numbers for your cameras.

With Docker

export SERVER_ARGS="--env ALOHA --default_prompt='toast out of toaster'"
docker compose -f examples/aloha_real/compose.yml up --build

Without Docker

Terminal window 1:

# Create virtual environment
uv venv --python 3.10 examples/aloha_real/.venv
source examples/aloha_real/.venv/bin/activate
uv pip sync examples/aloha_real/requirements.txt
uv pip install -e packages/openpi-client

# Run the robot
python examples/aloha_real/main.py

Terminal window 2:

roslaunch --wait aloha ros_nodes.launch

Terminal window 3:

uv run scripts/serve_policy.py --env ALOHA --default_prompt='toast out of toaster'

Model Guide

The Pi0 Base Model is an out-of-the-box model for general tasks. You can find more details in the technical report.

While we strongly recommend fine-tuning the model to your own data to adapt it to particular tasks, it may be possible to prompt the model to attempt some tasks that were in the pre-training data. For example, below is a video of the model attempting the "toast out of toaster" task.

Training on your own Aloha dataset

OpenPI suppports training on data collected in the default aloha hdf5 format using the scripts/aloha_hd5.py conversion script. Once the dataset is converted, add a new TrainConfig to src/openpi/training/configs.py (see the aloha_static_cups_open example config) and replace repo id with the id assigned to your dataset during conversion. Before training on a new dataset, you must first compute the norm stats using scripts/compute_norm_stats.py.

NOTE: When finetuning the pi0 base model on Aloha data it is recommended that you set adapt_to_pi=True. This maps the state and action spaces from the original aloha data to the state and action spaces of the aloha data used to train the base model.

2.5 KiB Raw Blame History