Skip to content

Fixes on robot integration tutorial #1290

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 17 additions & 16 deletions docs/source/integrate_hardware.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ If you're using Feetech or Dynamixel motors, LeRobot provides built-in bus inter
Please refer to the [`MotorsBus`](https://github.com/huggingface/lerobot/blob/main/lerobot/common/motors/motors_bus.py) abstract class to learn about its API.
For a good example of how it can be used, you can have a look at our own [SO101 follower implementation](https://github.com/huggingface/lerobot/blob/main/lerobot/common/robots/so101_follower/so101_follower.py)

Use these if compatible! Otherwise, you'll need to find or write a Python interface (not covered in this tutorial):
Use these if compatible. Otherwise, you'll need to find or write a Python interface (not covered in this tutorial):
- Find an existing SDK in Python (or use bindings to C/C++)
- Or implement a basic communication wrapper (e.g., via pyserial, socket, or CANopen)

Expand All @@ -32,7 +32,7 @@ For Feetech and Dynamixel, we currently support these servos:
- SCS series (protocol 1): `scs0009`
- Dynamixel (protocol 2.0 only): `xl330-m077`, `xl330-m288`, `xl430-w250`, `xm430-w350`, `xm540-w270`, `xc430-w150`

If you are using Feetech or Dynamixel servos that are not in this list, you can add those in the [Feetech table](https://github.com/huggingface/lerobot/blob/main/lerobot/common/motors/feetech/tables.py) or [Dynamixel table](https://github.com/huggingface/lerobot/blob/main/lerobot/common/motors/dynamixel/tables.py). Depending on the model, this will require you to add model-specific information. In most cases though, there should be a lot of additions to do.
If you are using Feetech or Dynamixel servos that are not in this list, you can add those in the [Feetech table](https://github.com/huggingface/lerobot/blob/main/lerobot/common/motors/feetech/tables.py) or [Dynamixel table](https://github.com/huggingface/lerobot/blob/main/lerobot/common/motors/dynamixel/tables.py). Depending on the model, this will require you to add model-specific information. In most cases though, there shouldn't be a lot of additions to do.

In the next sections, we'll use a `FeetechMotorsBus` as the motors interface for the examples. Replace it and adapt to your motors if necessary.

Expand Down Expand Up @@ -158,7 +158,7 @@ def is_connected(self) -> bool:

### `connect()`

This method should establish communication with the hardware. Moreover, if your robot needs calibration is not calibrated, it should start a calibration procedure by default. If your robot needs some specific configuration, this should also be called here.
This method should establish communication with the hardware. Moreover, if your robot needs calibration and is not calibrated, it should start a calibration procedure by default. If your robot needs some specific configuration, this should also be called here.

```python
def connect(self, calibrate: bool = True) -> None:
Expand Down Expand Up @@ -272,30 +272,31 @@ Returns a dictionary of sensor values from the robot. These typically include mo
```python
def get_observation(self) -> dict[str, Any]:
if not self.is_connected:
raise RuntimeError("Robot is not connected")
raise ConnectionError(f"{self} is not connected.")

joint_pos = self.motor_interface.read_joint_positions()
gripper = self.motor_interface.read_gripper_state()
image = self.camera.get_frame()
# Read arm position
obs_dict = self.bus.sync_read("Present_Position")
obs_dict = {f"{motor}.pos": val for motor, val in obs_dict.items()}

return {
"joint_positions": joint_pos,
"gripper_open": gripper,
"camera_image": image,
}
# Capture images from cameras
for cam_key, cam in self.cameras.items():
obs_dict[cam_key] = cam.async_read()

return obs_dict
```

### `send_action()`

Takes a dictionary that matches `action_features`, and sends it to your hardware. You can add safety limits (clipping, smoothing) and return what was actually sent.

For simplicity, we won't be adding any modification of the actions in our example here.

```python
def send_action(self, action: dict[str, Any]) -> dict[str, Any]:
if not self.is_connected:
raise RuntimeError("Robot is not connected")
goal_pos = {key.removesuffix(".pos"): val for key, val in action.items()}

self.motor_interface.set_joint_positions(action["joint_position_goals"])
self.motor_interface.set_gripper(action["gripper_command"])
# Send goal position to the arm
self.bus.sync_write("Goal_Position", goal_pos)

return action
```
Expand Down