* (unscrewing things up) (#2288)
* fix: expose a function explicitly building a frame for inference
* fix: first make dataset frame, then make ready for inference
* fix: reducing reliance on lerobot record for policy's ouptuts too
* fix: encapsulating squeezing out + device handling from predict action
* fix: remove duplicated call to build_inference_frame and add a function to only perform data type handling (whole conversion is: keys matching + data type conversion)
* refactor(envs): add custom-observation-size (#2167)
* fix: add MockMotorBus to MockRobot
* rl: first drafts
* add: all components of HIL SERL
* fix: actor block works
* fix: less friction, less friction
* add: hil-serl complete example
* fix: dataset names
* fix: restructuring example folder
* fix: act works but found bug in how ACT works
* fix: same path for both pre and postprocessors
* fix: paths
* add: example usage for act
* add: using ACT example
* fix: training examples
* fix: using examples
* fix: camera index
* fix: rename workflows into tutorial so that the path of the files is lerobot/examples/tutorial/...
* fix: upload everything in one repo
* fix: model name
* fix: simplify model path
* add: VLAs example
---------
Signed-off-by: Francesco Capuano <74058581+fracapuano@users.noreply.github.com>
* fix: minor fix using named attributes
* fix: change model to act
* fix: named attributes for inference frame building
* fix: minor fixes to smolvla
* fix: small changes to pi0
* remove: old file that should have never been committed (ups sorry sorry)
---------
Signed-off-by: Francesco Capuano <74058581+fracapuano@users.noreply.github.com>
Set camera width/height 1st before FPS setting, to avoid FPS set failure alike:
ERROR:__main__:Failed to connect or configure OpenCV camera /dev/video2: OpenCVCamera(/dev/video2) failed to set fps=30 (actual_fps=25.0).
* Enhance OpenCVCamera with FOURCC support and validation
- Added FOURCC configuration option to OpenCVCamera and OpenCVCameraConfig for specifying video format.
- Implemented _validate_fourcc method to validate and set the camera's FOURCC code.
- Updated _configure_capture_settings to apply FOURCC settings before FPS and resolution.
- Enhanced camera detection to include default FOURCC code in camera info.
- Updated documentation to reflect new FOURCC parameter and its implications on performance.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add tests for FOURCC configuration in OpenCVCamera
- Implemented tests to validate FOURCC configuration and its application in OpenCVCamera.
- Added checks for valid FOURCC codes and ensured that invalid codes raise appropriate errors.
- Included a test for camera connection functionality using specified FOURCC settings.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix circular import in __init__.py - change to relative import
* Update src/lerobot/cameras/opencv/configuration_opencv.py
Co-authored-by: Steven Palma <imstevenpmwork@ieee.org>
Signed-off-by: hls <56255627+forgetwhatuwant@users.noreply.github.com>
* Update src/lerobot/cameras/opencv/configuration_opencv.py
Co-authored-by: Steven Palma <imstevenpmwork@ieee.org>
Signed-off-by: hls <56255627+forgetwhatuwant@users.noreply.github.com>
* fix(camera_opencv): ensure MSMF hardware transform compatibility on Windows before importing OpenCV
* This change reverts the import from a relative import (.) back to the absolute import (lerobot.) as it was previously
* opencv/config: satisfy Ruff SIM102 by merging nested if for fourcc validation
* style(opencv/config): apply ruff-format changes
---------
Signed-off-by: hls <56255627+forgetwhatuwant@users.noreply.github.com>
Signed-off-by: Steven Palma <imstevenpmwork@ieee.org>
Co-authored-by: forgetwhatuwant <forgetwhatuwant@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Steven Palma <imstevenpmwork@ieee.org>
* fix: update policy handling and type annotations
added typehint and addressed the error of mypy
* fix: rename should_push_to_hub to push_to_hub
I find that there are other dependencies of push_to_hub so I fix the property name back to original one.
* fix: typo
* fix: changed the position of try-except block
As the copilot said, use raise before `hf_hub_download` would stop program even it is able to download
* fix: update pre-commit configuration and mypy settings
add args: --follow-imports=silent to pass error which have no relationship with src/lerobot/configs
* fix: remove the specific path in .pre-commit-config.yaml
* feat: enhance typehint to adapt mypy strict mode.
* fix: remove duplicate FileNotFoundError check in PreTrainedConfig
* fix: make "pre-commit run --all-files" pass
* fix: replace logging with logger for better logging practices
* fix: fixed extra changes of lint and format changes
* fix: fixed extra changes out of "configs" module
* Update src/lerobot/configs/policies.py
Co-authored-by: Steven Palma <imstevenpmwork@ieee.org>
Signed-off-by: tetsugo02 <131431116+tetsugo02@users.noreply.github.com>
* fix: add logging for scratch job
---------
Signed-off-by: Adil Zouitine <adilzouitinegm@gmail.com>
Signed-off-by: tetsugo02 <131431116+tetsugo02@users.noreply.github.com>
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
Co-authored-by: Steven Palma <imstevenpmwork@ieee.org>
* feat(mypy-compliant): Ensure the model module passes MyPy type checks
* fix
* uncomment pyproject.toml for model module
* fix
* fix
---------
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
Co-authored-by: Steven Palma <imstevenpmwork@ieee.org>
* refactor(env): introduce explicit gym ID handling in EnvConfig/factory
This commit introduces properties for the gym package/ID associated
with and environment config. They default to the current defaults
(`gym_{package_name}/{task_id}`) to avoid breaking changes, but allow
for easier use of external gym environments.
Subclasses of `EnvConfig` can override the default properties to allow
the factory to import (i.e. register) the gym env from a specific module,
and also instantiate the env from any ID string.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* more changes
* quality
* fix test
---------
Co-authored-by: Ben Sprenger <ben.sprenger@rogers.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
* Prevents resource leak in video_utils when getting width and height
Added the with statement when opening the image to ensure that the file handle is properly closed after its contents are read.
Otherwise, shutil.rmtree(img_dir) will fail when called after the encode_video_frames function completes.
Signed-off-by: Lycoris <32864669+lycoris1129@users.noreply.github.com>
---------
Signed-off-by: Lycoris <32864669+lycoris1129@users.noreply.github.com>
* Enhance training and logging functionality with accelerator support
- Added support for multi-GPU training by introducing an `accelerator` parameter in training functions.
- Updated `update_policy` to handle gradient updates based on the presence of an accelerator.
- Modified logging to prevent duplicate messages in non-main processes.
- Enhanced `set_seed` and `get_safe_torch_device` functions to accommodate accelerator usage.
- Updated `MetricsTracker` to account for the number of processes when calculating metrics.
- Introduced a new feature in `pyproject.toml` for the `accelerate` library dependency.
* Initialize logging in training script for both main and non-main processes
- Added `init_logging` calls to ensure proper logging setup when using the accelerator and in standard training mode.
- This change enhances the clarity and consistency of logging during training sessions.
* add docs and only push model once
* Place logging under accelerate and update docs
* fix pre commit
* only log in main process
* main logging
* try with local rank
* add tests
* change runner
* fix test
* dont push to hub in multi gpu tests
* pre download dataset in tests
* small fixes
* fix path optimizer state
* update docs, and small improvements in train
* simplify accelerate main process detection
* small improvements in train
* fix OOM bug
* change accelerate detection
* add some debugging
* always use accelerate
* cleanup update method
* cleanup
* fix bug
* scale lr decay if we reduce steps
* cleanup logging
* fix formatting
* encorperate feedback pr
* add min memory to cpu tests
* use accelerate to determin logging
* fix precommit and fix tests
* chore: minor details
---------
Co-authored-by: AdilZouitine <adilzouitinegm@gmail.com>
Co-authored-by: Steven Palma <steven.palma@huggingface.co>
