From 2f8d98b05e622c2b8dbbf931b601c44127f4124c Mon Sep 17 00:00:00 2001 From: Simon Alibert <75076266+aliberts@users.noreply.github.com> Date: Fri, 1 Aug 2025 17:39:39 +0200 Subject: [PATCH] Update readme (#1570) * Cleanup badges * Remove comment * Remove profiling section * Move acknowledgment * Move citations * Fix badge display * Move build your robot section * Fix nightly badge * Revert be13b3f * Update README.md Co-authored-by: HUANG TZU-CHUN Signed-off-by: Simon Alibert <75076266+aliberts@users.noreply.github.com> * chore(docs): optimize readme for PyPI rendering * chore(docs): move policy readme to docs folder + symlink in policy dirs * fix(docs): max width og lerobot logo + url in citation block --------- Signed-off-by: Simon Alibert <75076266+aliberts@users.noreply.github.com> Co-authored-by: HUANG TZU-CHUN Co-authored-by: Steven Palma --- README.md | 201 ++++------------------- docs/source/policy_act_README.md | 14 ++ docs/source/policy_diffusion_README.md | 14 ++ docs/source/policy_smolvla_README.md | 14 ++ docs/source/policy_tdmpc_README.md | 14 ++ docs/source/policy_vqbet_README.md | 14 ++ src/lerobot/policies/act/README.md | 1 + src/lerobot/policies/diffusion/README.md | 1 + src/lerobot/policies/smolvla/README.md | 1 + src/lerobot/policies/tdmpc/README.md | 1 + src/lerobot/policies/vqbet/README.md | 1 + 11 files changed, 108 insertions(+), 168 deletions(-) create mode 100644 docs/source/policy_act_README.md create mode 100644 docs/source/policy_diffusion_README.md create mode 100644 docs/source/policy_smolvla_README.md create mode 100644 docs/source/policy_tdmpc_README.md create mode 100644 docs/source/policy_vqbet_README.md create mode 120000 src/lerobot/policies/act/README.md create mode 120000 src/lerobot/policies/diffusion/README.md create mode 120000 src/lerobot/policies/smolvla/README.md create mode 120000 src/lerobot/policies/tdmpc/README.md create mode 120000 src/lerobot/policies/vqbet/README.md diff --git a/README.md b/README.md index 1d7cbcad..13cc95f9 100644 --- a/README.md +++ b/README.md @@ -1,25 +1,21 @@

- - - - LeRobot, Hugging Face Robotics Library - + LeRobot, Hugging Face Robotics Library

-[![Tests](https://github.com/huggingface/lerobot/actions/workflows/nightly-tests.yml/badge.svg?branch=main)](https://github.com/huggingface/lerobot/actions/workflows/nightly-tests.yml?query=branch%3Amain) -[![Coverage](https://codecov.io/gh/huggingface/lerobot/branch/main/graph/badge.svg?token=TODO)](https://codecov.io/gh/huggingface/lerobot) +[![Tests](https://github.com/huggingface/lerobot/actions/workflows/nightly.yml/badge.svg?branch=main)](https://github.com/huggingface/lerobot/actions/workflows/nighty.yml?query=branch%3Amain) [![Python versions](https://img.shields.io/pypi/pyversions/lerobot)](https://www.python.org/downloads/) [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://github.com/huggingface/lerobot/blob/main/LICENSE) [![Status](https://img.shields.io/pypi/status/lerobot)](https://pypi.org/project/lerobot/) [![Version](https://img.shields.io/pypi/v/lerobot)](https://pypi.org/project/lerobot/) -[![Examples](https://img.shields.io/badge/Examples-green.svg)](https://github.com/huggingface/lerobot/tree/main/examples) -[![Contributor Covenant](https://img.shields.io/badge/Contributor%20Covenant-v2.1%20adopted-ff69b4.svg)](https://github.com/huggingface/lerobot/blob/main/CODE_OF_CONDUCT.md) +[![Contributor Covenant](https://img.shields.io/badge/Contributor%20Covenant-v2.1-ff69b4.svg)](https://github.com/huggingface/lerobot/blob/main/CODE_OF_CONDUCT.md) [![Discord](https://dcbadge.vercel.app/api/server/C5P34WJ68S?style=flat)](https://discord.gg/s3KuuzsPFb) + +

@@ -29,10 +25,10 @@
HopeJR robot

Meet HopeJR – A humanoid robot arm and hand for dexterous manipulation!

@@ -51,20 +47,12 @@

-
- SO-101 follower arm - SO-101 leader arm -
+ + + + + +
SO-101 follower armSO-101 leader arm

Meet the updated SO100, the SO-101 – Just €114 per arm!

Train it in minutes with a few simple moves on your laptop.

@@ -76,7 +64,7 @@

Want to take it to the next level? Make your SO-101 mobile by building LeKiwi!

Check out the LeKiwi tutorial and bring your robot to life on wheels.

- LeKiwi mobile robot + LeKiwi mobile robot

@@ -99,9 +87,9 @@ - - - + + + @@ -110,24 +98,9 @@
ACT policy on ALOHA envTDMPC policy on SimXArm envDiffusion policy on PushT envACT policy on ALOHA envTDMPC policy on SimXArm envDiffusion policy on PushT env
ACT policy on ALOHA env
-### Acknowledgment - -- The LeRobot team 🤗 for building SmolVLA [Paper](https://arxiv.org/abs/2506.01844), [Blog](https://huggingface.co/blog/smolvla). -- Thanks to Tony Zhao, Zipeng Fu and colleagues for open sourcing ACT policy, ALOHA environments and datasets. Ours are adapted from [ALOHA](https://tonyzhaozh.github.io/aloha) and [Mobile ALOHA](https://mobile-aloha.github.io). -- Thanks to Cheng Chi, Zhenjia Xu and colleagues for open sourcing Diffusion policy, Pusht environment and datasets, as well as UMI datasets. Ours are adapted from [Diffusion Policy](https://diffusion-policy.cs.columbia.edu) and [UMI Gripper](https://umi-gripper.github.io). -- Thanks to Nicklas Hansen, Yunhai Feng and colleagues for open sourcing TDMPC policy, Simxarm environments and datasets. Ours are adapted from [TDMPC](https://github.com/nicklashansen/tdmpc) and [FOWM](https://www.yunhaifeng.com/FOWM). -- Thanks to Antonio Loquercio and Ashish Kumar for their early support. -- Thanks to [Seungjae (Jay) Lee](https://sjlee.cc/), [Mahi Shafiullah](https://mahis.life/) and colleagues for open sourcing [VQ-BeT](https://sjlee.cc/vq-bet/) policy and helping us adapt the codebase to our repository. The policy is adapted from [VQ-BeT repo](https://github.com/jayLEE0301/vq_bet_official). - ## Installation -Download our source code: - -```bash -git clone https://github.com/huggingface/lerobot.git -cd lerobot -``` - +LeRobot works with Python 3.10+ and PyTorch 2.2+. Create a virtual environment with Python 3.10 and activate it, e.g. with [`miniconda`](https://docs.anaconda.com/free/miniconda/index.html): ```bash @@ -154,7 +127,7 @@ conda install ffmpeg -c conda-forge Install 🤗 LeRobot: ```bash -pip install -e . +pip install lerobot ``` > **NOTE:** If you encounter build errors, you may need to install additional dependencies (`cmake`, `build-essential`, and `ffmpeg libs`). On Linux, run: @@ -182,7 +155,7 @@ wandb login ### Visualize datasets -Check out [example 1](./examples/1_load_lerobot_dataset.py) that illustrates how to use our dataset class which automatically downloads data from the Hugging Face hub. +Check out [example 1](https://github.com/huggingface/lerobot/blob/main/examples/1_load_lerobot_dataset.py) that illustrates how to use our dataset class which automatically downloads data from the Hugging Face hub. You can also locally visualize episodes from a dataset on the hub by executing our script from the command line: @@ -212,7 +185,7 @@ Our script can also visualize datasets stored on a distant server. See `python - A dataset in `LeRobotDataset` format is very simple to use. It can be loaded from a repository on the Hugging Face hub or a local folder simply with e.g. `dataset = LeRobotDataset("lerobot/aloha_static_coffee")` and can be indexed into like any Hugging Face and PyTorch dataset. For instance `dataset[0]` will retrieve a single temporal frame from the dataset containing observation(s) and an action as PyTorch tensors ready to be fed to a model. -A specificity of `LeRobotDataset` is that, rather than retrieving a single frame by its index, we can retrieve several frames based on their temporal relationship with the indexed frame, by setting `delta_timestamps` to a list of relative times with respect to the indexed frame. For example, with `delta_timestamps = {"observation.image": [-1, -0.5, -0.2, 0]}` one can retrieve, for a given index, 4 frames: 3 "previous" frames 1 second, 0.5 seconds, and 0.2 seconds before the indexed frame, and the indexed frame itself (corresponding to the 0 entry). See example [1_load_lerobot_dataset.py](examples/1_load_lerobot_dataset.py) for more details on `delta_timestamps`. +A specificity of `LeRobotDataset` is that, rather than retrieving a single frame by its index, we can retrieve several frames based on their temporal relationship with the indexed frame, by setting `delta_timestamps` to a list of relative times with respect to the indexed frame. For example, with `delta_timestamps = {"observation.image": [-1, -0.5, -0.2, 0]}` one can retrieve, for a given index, 4 frames: 3 "previous" frames 1 second, 0.5 seconds, and 0.2 seconds before the indexed frame, and the indexed frame itself (corresponding to the 0 entry). See example [1_load_lerobot_dataset.py](https://github.com/huggingface/lerobot/blob/main/examples/1_load_lerobot_dataset.py) for more details on `delta_timestamps`. Under the hood, the `LeRobotDataset` format makes use of several ways to serialize data which can be useful to understand if you plan to work more closely with this format. We tried to make a flexible yet simple dataset format that would cover most type of features and specificities present in reinforcement learning and robotics, in simulation and in real-world, with a focus on cameras and robot states but easily extended to other types of sensory inputs as long as they can be represented by a tensor. @@ -256,7 +229,7 @@ Dataset can be uploaded/downloaded from the HuggingFace hub seamlessly. To work ### Evaluate a pretrained policy -Check out [example 2](./examples/2_evaluate_pretrained_policy.py) that illustrates how to download a pretrained policy from Hugging Face hub, and run an evaluation on its corresponding environment. +Check out [example 2](https://github.com/huggingface/lerobot/blob/main/examples/2_evaluate_pretrained_policy.py) that illustrates how to download a pretrained policy from Hugging Face hub, and run an evaluation on its corresponding environment. We also provide a more capable script to parallelize the evaluation over multiple environments during the same rollout. Here is an example with a pretrained model hosted on [lerobot/diffusion_pusht](https://huggingface.co/lerobot/diffusion_pusht): @@ -280,13 +253,13 @@ See `python -m lerobot.scripts.eval --help` for more instructions. ### Train your own policy -Check out [example 3](./examples/3_train_policy.py) that illustrates how to train a model using our core library in python, and [example 4](./examples/4_train_policy_with_script.md) that shows how to use our training script from command line. +Check out [example 3](https://github.com/huggingface/lerobot/blob/main/examples/3_train_policy.py) that illustrates how to train a model using our core library in python, and [example 4](https://github.com/huggingface/lerobot/blob/main/examples/4_train_policy_with_script.md) that shows how to use our training script from command line. To use wandb for logging training and evaluation curves, make sure you've run `wandb login` as a one-time setup step. Then, when running the training command above, enable WandB in the configuration by adding `--wandb.enable=true`. -A link to the wandb logs for the run will also show up in yellow in your terminal. Here is an example of what they look like in your browser. Please also check [here](./examples/4_train_policy_with_script.md#typical-logs-and-metrics) for the explanation of some commonly used metrics in logs. +A link to the wandb logs for the run will also show up in yellow in your terminal. Here is an example of what they look like in your browser. Please also check [here](https://github.com/huggingface/lerobot/blob/main/examples/4_train_policy_with_script.md#typical-logs-and-metrics) for the explanation of some commonly used metrics in logs. -![](media/wandb.png) +\WandB logs example Note: For efficiency, during training every checkpoint is evaluated on a low number of episodes. You may use `--eval.n_episodes=500` to evaluate on more episodes than the default. Or, after training, you may want to re-evaluate your best checkpoints on more episodes or change the evaluation settings. See `python -m lerobot.scripts.eval --help` for more instructions. @@ -305,26 +278,6 @@ reproduces SOTA results for Diffusion Policy on the PushT task. If you would like to contribute to 🤗 LeRobot, please check out our [contribution guide](https://github.com/huggingface/lerobot/blob/main/CONTRIBUTING.md). - - ### Add a pretrained policy Once you have trained a policy you may upload it to the Hugging Face hub using a hub id that looks like `${hf_user}/${repo_name}` (e.g. [lerobot/diffusion_pusht](https://huggingface.co/lerobot/diffusion_pusht)). @@ -341,34 +294,16 @@ To upload these to the hub, run the following: huggingface-cli upload ${hf_user}/${repo_name} path/to/pretrained_model ``` -See [eval.py](https://github.com/huggingface/lerobot/blob/main/lerobot/scripts/eval.py) for an example of how other people may use your policy. +See [eval.py](https://github.com/huggingface/lerobot/blob/main/src/lerobot/scripts/eval.py) for an example of how other people may use your policy. -### Improve your code with profiling +### Acknowledgment -An example of a code snippet to profile the evaluation of a policy: - - -```python -from torch.profiler import profile, record_function, ProfilerActivity - -def trace_handler(prof): - prof.export_chrome_trace(f"tmp/trace_schedule_{prof.step_num}.json") - -with profile( - activities=[ProfilerActivity.CPU, ProfilerActivity.CUDA], - schedule=torch.profiler.schedule( - wait=2, - warmup=2, - active=3, - ), - on_trace_ready=trace_handler -) as prof: - with record_function("eval_policy"): - for i in range(num_episodes): - prof.step() - # insert code to profile, potentially whole body of eval_policy function -``` - +- The LeRobot team 🤗 for building SmolVLA [Paper](https://arxiv.org/abs/2506.01844), [Blog](https://huggingface.co/blog/smolvla). +- Thanks to Tony Zhao, Zipeng Fu and colleagues for open sourcing ACT policy, ALOHA environments and datasets. Ours are adapted from [ALOHA](https://tonyzhaozh.github.io/aloha) and [Mobile ALOHA](https://mobile-aloha.github.io). +- Thanks to Cheng Chi, Zhenjia Xu and colleagues for open sourcing Diffusion policy, Pusht environment and datasets, as well as UMI datasets. Ours are adapted from [Diffusion Policy](https://diffusion-policy.cs.columbia.edu) and [UMI Gripper](https://umi-gripper.github.io). +- Thanks to Nicklas Hansen, Yunhai Feng and colleagues for open sourcing TDMPC policy, Simxarm environments and datasets. Ours are adapted from [TDMPC](https://github.com/nicklashansen/tdmpc) and [FOWM](https://www.yunhaifeng.com/FOWM). +- Thanks to Antonio Loquercio and Ashish Kumar for their early support. +- Thanks to [Seungjae (Jay) Lee](https://sjlee.cc/), [Mahi Shafiullah](https://mahis.life/) and colleagues for open sourcing [VQ-BeT](https://sjlee.cc/vq-bet/) policy and helping us adapt the codebase to our repository. The policy is adapted from [VQ-BeT repo](https://github.com/jayLEE0301/vq_bet_official). ## Citation @@ -383,76 +318,6 @@ If you want, you can cite this work with: } ``` -Additionally, if you are using any of the particular policy architecture, pretrained models, or datasets, it is recommended to cite the original authors of the work as they appear below: - -- [SmolVLA](https://arxiv.org/abs/2506.01844) - -```bibtex -@article{shukor2025smolvla, - title={SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Robotics}, - author={Shukor, Mustafa and Aubakirova, Dana and Capuano, Francesco and Kooijmans, Pepijn and Palma, Steven and Zouitine, Adil and Aractingi, Michel and Pascal, Caroline and Russi, Martino and Marafioti, Andres and Alibert, Simon and Cord, Matthieu and Wolf, Thomas and Cadene, Remi}, - journal={arXiv preprint arXiv:2506.01844}, - year={2025} -} -``` - -- [Diffusion Policy](https://diffusion-policy.cs.columbia.edu) - -```bibtex -@article{chi2024diffusionpolicy, - author = {Cheng Chi and Zhenjia Xu and Siyuan Feng and Eric Cousineau and Yilun Du and Benjamin Burchfiel and Russ Tedrake and Shuran Song}, - title ={Diffusion Policy: Visuomotor Policy Learning via Action Diffusion}, - journal = {The International Journal of Robotics Research}, - year = {2024}, -} -``` - -- [ACT or ALOHA](https://tonyzhaozh.github.io/aloha) - -```bibtex -@article{zhao2023learning, - title={Learning fine-grained bimanual manipulation with low-cost hardware}, - author={Zhao, Tony Z and Kumar, Vikash and Levine, Sergey and Finn, Chelsea}, - journal={arXiv preprint arXiv:2304.13705}, - year={2023} -} -``` - -- [TDMPC](https://www.nicklashansen.com/td-mpc/) - -```bibtex -@inproceedings{Hansen2022tdmpc, - title={Temporal Difference Learning for Model Predictive Control}, - author={Nicklas Hansen and Xiaolong Wang and Hao Su}, - booktitle={ICML}, - year={2022} -} -``` - -- [VQ-BeT](https://sjlee.cc/vq-bet/) - -```bibtex -@article{lee2024behavior, - title={Behavior generation with latent actions}, - author={Lee, Seungjae and Wang, Yibin and Etukuru, Haritheja and Kim, H Jin and Shafiullah, Nur Muhammad Mahi and Pinto, Lerrel}, - journal={arXiv preprint arXiv:2403.03181}, - year={2024} -} -``` - -- [HIL-SERL](https://hil-serl.github.io/) - -```bibtex -@Article{luo2024hilserl, -title={Precise and Dexterous Robotic Manipulation via Human-in-the-Loop Reinforcement Learning}, -author={Jianlan Luo and Charles Xu and Jeffrey Wu and Sergey Levine}, -year={2024}, -eprint={2410.21845}, -archivePrefix={arXiv}, -primaryClass={cs.RO} -} -``` - ## Star History [![Star History Chart](https://api.star-history.com/svg?repos=huggingface/lerobot&type=Timeline)](https://star-history.com/#huggingface/lerobot&Timeline) diff --git a/docs/source/policy_act_README.md b/docs/source/policy_act_README.md new file mode 100644 index 00000000..371a9136 --- /dev/null +++ b/docs/source/policy_act_README.md @@ -0,0 +1,14 @@ +## Paper + +https://tonyzhaozh.github.io/aloha + +## Citation + +```bibtex +@article{zhao2023learning, + title={Learning fine-grained bimanual manipulation with low-cost hardware}, + author={Zhao, Tony Z and Kumar, Vikash and Levine, Sergey and Finn, Chelsea}, + journal={arXiv preprint arXiv:2304.13705}, + year={2023} +} +``` diff --git a/docs/source/policy_diffusion_README.md b/docs/source/policy_diffusion_README.md new file mode 100644 index 00000000..9ec934ad --- /dev/null +++ b/docs/source/policy_diffusion_README.md @@ -0,0 +1,14 @@ +## Paper + +https://diffusion-policy.cs.columbia.edu + +## Citation + +```bibtex +@article{chi2024diffusionpolicy, + author = {Cheng Chi and Zhenjia Xu and Siyuan Feng and Eric Cousineau and Yilun Du and Benjamin Burchfiel and Russ Tedrake and Shuran Song}, + title ={Diffusion Policy: Visuomotor Policy Learning via Action Diffusion}, + journal = {The International Journal of Robotics Research}, + year = {2024}, +} +``` diff --git a/docs/source/policy_smolvla_README.md b/docs/source/policy_smolvla_README.md new file mode 100644 index 00000000..ee567ee8 --- /dev/null +++ b/docs/source/policy_smolvla_README.md @@ -0,0 +1,14 @@ +## Paper + +https://arxiv.org/abs/2506.01844 + +## Citation + +```bibtex +@article{shukor2025smolvla, + title={SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Robotics}, + author={Shukor, Mustafa and Aubakirova, Dana and Capuano, Francesco and Kooijmans, Pepijn and Palma, Steven and Zouitine, Adil and Aractingi, Michel and Pascal, Caroline and Russi, Martino and Marafioti, Andres and Alibert, Simon and Cord, Matthieu and Wolf, Thomas and Cadene, Remi}, + journal={arXiv preprint arXiv:2506.01844}, + year={2025} +} +``` diff --git a/docs/source/policy_tdmpc_README.md b/docs/source/policy_tdmpc_README.md new file mode 100644 index 00000000..804f166c --- /dev/null +++ b/docs/source/policy_tdmpc_README.md @@ -0,0 +1,14 @@ +## Paper + +https://www.nicklashansen.com/td-mpc/ + +## Citation + +```bibtex +@inproceedings{Hansen2022tdmpc, + title={Temporal Difference Learning for Model Predictive Control}, + author={Nicklas Hansen and Xiaolong Wang and Hao Su}, + booktitle={ICML}, + year={2022} +} +``` diff --git a/docs/source/policy_vqbet_README.md b/docs/source/policy_vqbet_README.md new file mode 100644 index 00000000..02f95b7c --- /dev/null +++ b/docs/source/policy_vqbet_README.md @@ -0,0 +1,14 @@ +## Paper + +https://sjlee.cc/vq-bet/ + +## Citation + +```bibtex +@article{lee2024behavior, + title={Behavior generation with latent actions}, + author={Lee, Seungjae and Wang, Yibin and Etukuru, Haritheja and Kim, H Jin and Shafiullah, Nur Muhammad Mahi and Pinto, Lerrel}, + journal={arXiv preprint arXiv:2403.03181}, + year={2024} +} +``` diff --git a/src/lerobot/policies/act/README.md b/src/lerobot/policies/act/README.md new file mode 120000 index 00000000..04602009 --- /dev/null +++ b/src/lerobot/policies/act/README.md @@ -0,0 +1 @@ +../../../../docs/source/policy_act_README.md \ No newline at end of file diff --git a/src/lerobot/policies/diffusion/README.md b/src/lerobot/policies/diffusion/README.md new file mode 120000 index 00000000..d332d79c --- /dev/null +++ b/src/lerobot/policies/diffusion/README.md @@ -0,0 +1 @@ +../../../../docs/source/policy_diffusion_README.md \ No newline at end of file diff --git a/src/lerobot/policies/smolvla/README.md b/src/lerobot/policies/smolvla/README.md new file mode 120000 index 00000000..f8de4026 --- /dev/null +++ b/src/lerobot/policies/smolvla/README.md @@ -0,0 +1 @@ +../../../../docs/source/policy_smolvla_README.md \ No newline at end of file diff --git a/src/lerobot/policies/tdmpc/README.md b/src/lerobot/policies/tdmpc/README.md new file mode 120000 index 00000000..413ea87b --- /dev/null +++ b/src/lerobot/policies/tdmpc/README.md @@ -0,0 +1 @@ +../../../../docs/source/policy_tdmpc_README.md \ No newline at end of file diff --git a/src/lerobot/policies/vqbet/README.md b/src/lerobot/policies/vqbet/README.md new file mode 120000 index 00000000..a4ae9291 --- /dev/null +++ b/src/lerobot/policies/vqbet/README.md @@ -0,0 +1 @@ +../../../../docs/source/policy_vqbet_README.md \ No newline at end of file