tangger/lerobot
1
0
Fork 1
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
756 Commits 153 Branches 0 Tags
ff47c0b0d37dae20ae4f82437dab7774db8ae128
Commit Graph

756 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Michel Aractingi
ff47c0b0d3 - Fixed big issue in the loading of the policy parameters sent by the learner to the actor -- pass only the actor to the update_policy_parameters and remove strict=False 
- Fixed big issue in the normalization of the actions in the `forward` function of the critic -- remove the `torch.no_grad` decorator in `normalize.py` in the normalization function
- Fixed performance issue to boost the optimization frequency by setting the storage device to be the same as the device of learning.

Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
 2025-02-19 16:22:51 +00:00
AdilZouitine
befa1fe9af Re-enable parameter push thread in learner server 
- Uncomment and start the param_push_thread
- Restore thread joining for param_push_thread
 2025-02-17 10:26:33 +00:00
AdilZouitine
446f434a8e Improve wandb logging and custom step tracking in logger 
- Modify logger to support multiple custom step keys
- Update logging method to handle custom step keys more flexibly

- Enhance logging of optimization step and frequency
Co-authored-by: michel-aractingi  <michel.aractingi@gmail.com>
 2025-02-17 10:08:49 +00:00
AdilZouitine
2f3370e42f Add maniskill support. 
Co-authored-by: Michel Aractingi <michel.aractingi@gmail.com>
 2025-02-14 19:53:29 +00:00
Michel Aractingi
7ae368e983 Fixed bug in the action scale of the intervention actions and offline dataset actions. (scale by inverse delta) 
Co-authored-by: Adil Zouitine <adizouitinegm@gmail.com>
 2025-02-14 15:17:16 +01:00
Michel Aractingi
36711d766a Modified crop_dataset_roi interface to automatically write the cropped parameters to a json file in the meta of the dataset 
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
 2025-02-14 12:32:45 +01:00
Michel Aractingi
c9e50bb9b1 Optimized the replay buffer from the memory side to store data on cpu instead of a gpu device and send the batches to the gpu. 
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
 2025-02-13 18:03:57 +01:00
Michel Aractingi
95de8e273d nit 
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
 2025-02-13 17:12:57 +01:00
Michel Aractingi
b07d95f0dd removed uncomment in actor server 
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
 2025-02-13 16:53:33 +01:00
Michel Aractingi
d9a70376d8 Changed the init_final value to center the starting mean and std of the policy 
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
 2025-02-13 16:42:43 +01:00
Michel Aractingi
0c32008466 Changed bounds for a new so100 robot 
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
 2025-02-13 15:43:30 +01:00
Michel Aractingi
c462a478c7 Hardcoded some normalization parameters. TODO refactor 
Added masking actions on the level of the intervention actions and offline dataset

Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
 2025-02-13 14:27:14 +01:00
Michel Aractingi
459f22ed30 fix log_alpha in modeling_sac: change to nn.parameter 
added pretrained vision model in policy

Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
 2025-02-13 11:26:24 +01:00
Michel Aractingi
dc086dc21f Added logging for interventions to monitor the rate of interventions through time 
Added an s keyboard command to force success in the case the reward classifier fails

Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
 2025-02-13 11:04:49 +01:00
Michel Aractingi
b9217b06db Added possiblity to record and replay delta actions during teleoperation rather than absolute actions 
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
 2025-02-12 19:25:41 +01:00
Yoel
6868c88ef1 [PORT-Hilserl] classifier fixes (#695) 
Co-authored-by: Michel Aractingi <michel.aractingi@huggingface.co>
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
 2025-02-11 11:39:17 +01:00
Eugene Mironov
a1d16fb400 [Port HIL-SERL] Add resnet-10 as default encoder for HIL-SERL (#696) 
Co-authored-by: Khalil Meftah <kmeftah.khalil@gmail.com>
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
Co-authored-by: Michel Aractingi <michel.aractingi@huggingface.co>
Co-authored-by: Ke Wang <superwk1017@gmail.com>
 2025-02-11 11:37:00 +01:00
Michel Aractingi
a7db3959f5 - Added JointMaskingActionSpace wrapper in gym_manipulator in order to select which joints will be controlled. For example, we can disable the gripper actions for some tasks. 
- Added Nan detection mechanisms in the actor, learner and gym_manipulator for the case where we encounter nans in the loop.
- changed the non-blocking in the `.to(device)` functions to only work for the case of cuda because they were causing nans when running the policy on mps
- Added some joint clipping and limits in the env, robot and policy configs. TODO clean this part and make the limits in one config file only.

Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
 2025-02-11 11:34:46 +01:00
Michel Aractingi
b5f89439ff Added sac_real config file in the policym configs dir. 
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
 2025-02-10 16:08:13 +01:00
Michel Aractingi
d51374ce12 Several fixes to move the actor_server and learner_server code from the maniskill environment to the real robot environment. 
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
 2025-02-10 16:03:39 +01:00
Eugene Mironov
b63738674c [HIL-SERL port] Add Reward classifier benchmark tracking to chose best visual encoder (#688) 2025-02-06 18:39:51 +01:00
Michel Aractingi
12525242ce - Added lerobot/scripts/server/gym_manipulator.py that contains all the necessary wrappers to run a gym-style env around the real robot. 
- Added `lerobot/scripts/server/find_joint_limits.py` to test the min and max angles of the motion you wish the robot to explore during RL training.
- Added logic in `manipulator.py` to limit the maximum possible joint angles to allow motion within a predefined joint position range. The limits are specified in the yaml config for each robot. Checkout the so100.yaml.

Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
 2025-02-06 16:29:37 +01:00
Michel Aractingi
7d5a9530f7 fixed bug in crop_dataset_roi.py 
added missing buffer.pt in server dir

Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
 2025-02-05 18:22:50 +00:00
Michel Aractingi
e0527b4a6b Added additional wrappers for the environment: Action repeat, keyboard interface, reset wrapper 
Tested the reset mechanism and keyboard interface and the convert wrapper on the robots.

Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
 2025-02-04 17:41:14 +00:00
Michel Aractingi
efb1982eec Added crop_dataset_roi.py that allows you to load a lerobotdataset -> crop its images -> create a new lerobot dataset with the cropped and resized images. 
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
 2025-02-03 17:48:35 +00:00
Michel Aractingi
2211209be5 - Added base gym env class for the real robot environment. 
- Added several wrappers around the base gym env robot class.
- Including: time limit, reward classifier, crop images, preprocess observations.
- Added an interactive script crop_roi.py where the user can interactively select the roi in the observation images and return the correct crop values that will improve the policy and reward classifier performance.

Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
 2025-02-03 15:07:59 +00:00
Michel Aractingi
506821c7df - Refactor observation encoder in modeling_sac.py 
- added `torch.compile` to the actor and learner servers.
- organized imports in `train_sac.py`
- optimized the parameters push by not sending the frozen pre-trained encoder.

Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
 2025-02-03 15:07:58 +00:00
Yoel
f1c8bfe01e [Port HIL-SERL] Add HF vision encoder option in SAC (#651) 
Added support with custom pretrained vision encoder to the modeling sac implementation. Great job @ChorntonYoel !
 2025-02-03 15:07:58 +00:00
Michel Aractingi
7c89bd1018 Cleaned learner_server.py. Added several block function to improve readability. 
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
 2025-02-03 15:07:58 +00:00
Michel Aractingi
367dfe51c6 Added support for checkpointing the policy. We can save and load the policy state dict, optimizers state, optimization step and interaction step 
Added functions for converting the replay buffer from and to LeRobotDataset. When we want to save the replay buffer, we convert it first to LeRobotDataset format and save it locally and vice-versa.

Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
 2025-02-03 15:07:58 +00:00
Michel Aractingi
e856ffc91e Removed unnecessary time.sleep in the streaming server on the learner side 
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
 2025-02-03 15:07:58 +00:00
Michel Aractingi
9aabe212ea Added missing config files env/maniskill_example.yaml and policy/sac_maniskill.yaml that are necessary to run the lerobot implementation of sac with the maniskill baselines. 
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
 2025-02-03 15:07:58 +00:00
Michel Aractingi
42618f4bd6 - Added additional logging information in wandb around the timings of the policy loop and optimization loop. 
- Optimized critic design that improves the performance of the learner loop by a factor of 2
- Cleaned the code and fixed style issues

- Completed the config with actor_learner_config field that contains host-ip and port elemnts that are necessary for the actor-learner servers.

Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
 2025-02-03 15:07:58 +00:00
Michel Aractingi
36576c958f FREEDOM, added back the optimization loop code in learner_server.py 
Ran experiment with pushcube env from maniskill. The learning seem to work.

Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
 2025-02-03 15:07:58 +00:00
Michel Aractingi
322a78a378 Added server directory in lerobot/scripts that contains scripts and the protobuf message types to split training into two processes, acting and learning. The actor rollouts the policy and collects interaction data while the learner recieves the data, trains the policy and sends the updated parameters to the actor. The two scripts are ran simultaneously 
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
 2025-02-03 15:07:58 +00:00
AdilZouitine
d75b44f89f Stable version of rlpd + drq 2025-02-03 15:07:57 +00:00
AdilZouitine
1fb03d4cf2 Add type annotations and restructure SACConfig class fields 2025-02-03 15:07:57 +00:00
Adil Zouitine
7d2970fdfe Change SAC policy implementation with configuration and modeling classes 2025-02-03 15:07:50 +00:00
Adil Zouitine
8105efb338 Add rlpd tricks 2025-02-03 15:06:18 +00:00
Adil Zouitine
c1d4bf4b63 SAC works 2025-02-03 15:06:18 +00:00
Adil Zouitine
86df8a433d remove breakpoint 2025-02-03 15:06:18 +00:00
Adil Zouitine
956c547254 [WIP] correct sac implementation 2025-02-03 15:06:18 +00:00
Adil Zouitine
be965019bd Add rlpd tricks 2025-02-03 15:06:18 +00:00
Adil Zouitine
a0a50de8c9 SAC works 2025-02-03 15:06:18 +00:00
Adil Zouitine
c86dace4c2 remove breakpoint 2025-02-03 15:06:18 +00:00
Adil Zouitine
472a7f58ad [WIP] correct sac implementation 2025-02-03 15:06:14 +00:00
Pradeep Kadubandi
068efce3f8 Fix for the issue https://github.com/huggingface/lerobot/issues/638 (#639) 2025-02-03 15:04:03 +00:00
Philip Fung
df7310ea40 fixes to SO-100 readme (#600) 
Co-authored-by: Philip Fung <no@one>
Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com>
 2025-02-03 15:04:03 +00:00
Mishig
100f54ee07 [viz] Fixes & updates to html visualizer (#617) 2025-02-03 15:04:03 +00:00
CharlesCNorton
c2f7af3339 typo fix: batch_convert_dataset_v1_to_v2.py (#615) 
Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com>
 2025-02-03 15:04:03 +00:00