diff --git a/docs/source/integrate_hardware.mdx b/docs/source/integrate_hardware.mdx index 83db273d..f7de1cec 100644 --- a/docs/source/integrate_hardware.mdx +++ b/docs/source/integrate_hardware.mdx @@ -20,7 +20,7 @@ If you're using Feetech or Dynamixel motors, LeRobot provides built-in bus inter Please refer to the [`MotorsBus`](https://github.com/huggingface/lerobot/blob/main/lerobot/common/motors/motors_bus.py) abstract class to learn about its API. For a good example of how it can be used, you can have a look at our own [SO101 follower implementation](https://github.com/huggingface/lerobot/blob/main/lerobot/common/robots/so101_follower/so101_follower.py) -Use these if compatible! Otherwise, you'll need to find or write a Python interface (not covered in this tutorial): +Use these if compatible. Otherwise, you'll need to find or write a Python interface (not covered in this tutorial): - Find an existing SDK in Python (or use bindings to C/C++) - Or implement a basic communication wrapper (e.g., via pyserial, socket, or CANopen) @@ -32,7 +32,7 @@ For Feetech and Dynamixel, we currently support these servos: - SCS series (protocol 1): `scs0009` - Dynamixel (protocol 2.0 only): `xl330-m077`, `xl330-m288`, `xl430-w250`, `xm430-w350`, `xm540-w270`, `xc430-w150` -If you are using Feetech or Dynamixel servos that are not in this list, you can add those in the [Feetech table](https://github.com/huggingface/lerobot/blob/main/lerobot/common/motors/feetech/tables.py) or [Dynamixel table](https://github.com/huggingface/lerobot/blob/main/lerobot/common/motors/dynamixel/tables.py). Depending on the model, this will require you to add model-specific information. In most cases though, there should be a lot of additions to do. +If you are using Feetech or Dynamixel servos that are not in this list, you can add those in the [Feetech table](https://github.com/huggingface/lerobot/blob/main/lerobot/common/motors/feetech/tables.py) or [Dynamixel table](https://github.com/huggingface/lerobot/blob/main/lerobot/common/motors/dynamixel/tables.py). Depending on the model, this will require you to add model-specific information. In most cases though, there shouldn't be a lot of additions to do. In the next sections, we'll use a `FeetechMotorsBus` as the motors interface for the examples. Replace it and adapt to your motors if necessary. @@ -158,7 +158,7 @@ def is_connected(self) -> bool: ### `connect()` -This method should establish communication with the hardware. Moreover, if your robot needs calibration is not calibrated, it should start a calibration procedure by default. If your robot needs some specific configuration, this should also be called here. +This method should establish communication with the hardware. Moreover, if your robot needs calibration and is not calibrated, it should start a calibration procedure by default. If your robot needs some specific configuration, this should also be called here. ```python def connect(self, calibrate: bool = True) -> None: @@ -272,30 +272,31 @@ Returns a dictionary of sensor values from the robot. These typically include mo ```python def get_observation(self) -> dict[str, Any]: if not self.is_connected: - raise RuntimeError("Robot is not connected") + raise ConnectionError(f"{self} is not connected.") - joint_pos = self.motor_interface.read_joint_positions() - gripper = self.motor_interface.read_gripper_state() - image = self.camera.get_frame() + # Read arm position + obs_dict = self.bus.sync_read("Present_Position") + obs_dict = {f"{motor}.pos": val for motor, val in obs_dict.items()} - return { - "joint_positions": joint_pos, - "gripper_open": gripper, - "camera_image": image, - } + # Capture images from cameras + for cam_key, cam in self.cameras.items(): + obs_dict[cam_key] = cam.async_read() + + return obs_dict ``` ### `send_action()` Takes a dictionary that matches `action_features`, and sends it to your hardware. You can add safety limits (clipping, smoothing) and return what was actually sent. +For simplicity, we won't be adding any modification of the actions in our example here. + ```python def send_action(self, action: dict[str, Any]) -> dict[str, Any]: - if not self.is_connected: - raise RuntimeError("Robot is not connected") + goal_pos = {key.removesuffix(".pos"): val for key, val in action.items()} - self.motor_interface.set_joint_positions(action["joint_position_goals"]) - self.motor_interface.set_gripper(action["gripper_command"]) + # Send goal position to the arm + self.bus.sync_write("Goal_Position", goal_pos) return action ```