Skip to content

(feature): Add support for koch screwdriver robot and teleoperator #1303

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
64 commits
Select commit Hold shift + click to select a range
a729530
copy and rename files
jackvial Jun 14, 2025
bb6d0e6
Updated config to work for screwdriver
jackvial Jun 14, 2025
9e86057
fix new robot name
jackvial Jun 14, 2025
ce39515
Setup calibration, working on teleoperation
jackvial Jun 14, 2025
d1df712
Create koch screwdriver leader
jackvial Jun 14, 2025
7d31e14
comments
jackvial Jun 14, 2025
fd294d0
map the leader gripper position to follower screwdriver velocity
jackvial Jun 14, 2025
b4493f1
flip control direction
jackvial Jun 14, 2025
f3e8986
try fix elbow flex driver mode
jackvial Jun 14, 2025
70d826f
fix dictionary comprehension
jackvial Jun 14, 2025
29a8d35
invert screwdriver control direction
jackvial Jun 14, 2025
aa57e0e
comments
jackvial Jun 14, 2025
2e1eda2
make the screwdriver go faster
jackvial Jun 14, 2025
8e1b094
added record script
jackvial Jun 14, 2025
0845dc0
import the new robot in record
jackvial Jun 14, 2025
5323b4d
updated record scipt
jackvial Jun 14, 2025
699dae0
working on record
jackvial Jun 14, 2025
bdb2813
revert control utils changes
jackvial Jun 14, 2025
661f005
Add temp logging
jackvial Jun 14, 2025
0c98ec6
add third camera
jackvial Jun 14, 2025
166e9f0
video6
jackvial Jun 14, 2025
6fa08f1
added temp visualize script
jackvial Jun 14, 2025
e07e1a4
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 15, 2025
71db07d
refine prompt
jackvial Jun 15, 2025
30839c1
updated record script
jackvial Jun 15, 2025
5bca811
Merge branch 'jv/lerobot-hackathon-2025-screwdriver' of github.com:ja…
jackvial Jun 15, 2025
d890405
Updated docs
jackvial Jun 21, 2025
3cb3f08
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 21, 2025
9dba5f7
remove todo
jackvial Jun 21, 2025
17aad17
Format docs
jackvial Jun 21, 2025
400ccb7
fix conflict
jackvial Jun 21, 2025
74c7d48
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 21, 2025
e8a289b
remove my test scripts
jackvial Jun 21, 2025
27adaa9
Merge branch 'jv/lerobot-hackathon-2025-screwdriver' of github.com:ja…
jackvial Jun 21, 2025
d075531
Add docs about the camera config
jackvial Jun 21, 2025
84663d0
Added notes on feature scaling
jackvial Jun 21, 2025
1780d86
Added more comments
jackvial Jun 22, 2025
6d8d6ab
mapping explainer
jackvial Jun 22, 2025
21a626f
Clean up mapping comments
jackvial Jun 22, 2025
fc80228
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 22, 2025
fe97438
remove todos and debug logs
jackvial Jun 22, 2025
38b17c9
More tidy up
jackvial Jun 22, 2025
c366038
Merge branch 'jv/lerobot-hackathon-2025-screwdriver' of github.com:ja…
jackvial Jun 22, 2025
a3d6234
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 22, 2025
740a871
Remove pynput hacks
jackvial Jun 22, 2025
b5ae016
Merge branch 'jv/lerobot-hackathon-2025-screwdriver' of github.com:ja…
jackvial Jun 22, 2025
adb0069
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 22, 2025
6ce1cdf
fix naming
jackvial Jun 22, 2025
99e79ab
Added retries back
jackvial Jun 22, 2025
9c69a9c
Merge branch 'jv/lerobot-hackathon-2025-screwdriver' of github.com:ja…
jackvial Jun 22, 2025
6a7cf06
Allow deferring video encoding to after episode is record. I will che…
jackvial Jun 28, 2025
aea967d
Implemented clutch and backoff
jackvial Jun 29, 2025
11ce6c5
Added train val scripts
jackvial Jul 2, 2025
3308672
Log train loss in eval mode so we get the loss without dropout
jackvial Jul 2, 2025
a8d267d
kfold and sim docs
jackvial Jul 2, 2025
6a2c691
Updated inference script
jackvial Jul 3, 2025
70b7905
Adjusted clutch
jackvial Jul 6, 2025
5eeca5a
remove train eval
jackvial Jul 7, 2025
ffdbaed
Remove logging
jackvial Jul 10, 2025
d71f3ba
Cleanup
jackvial Jul 10, 2025
78cce09
more cleanup
jackvial Jul 10, 2025
b1b3a29
Remove koch leader invert
jackvial Jul 14, 2025
96b3410
Fixed conflicts
jackvial Jul 15, 2025
36c67c6
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 16, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions lerobot/common/robots/koch_screwdriver_follower/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
from .config_koch_screwdriver_follower import KochScrewdriverFollowerConfig
from .koch_screwdriver_follower import KochScrewdriverFollower
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
# Copyright 2024 The HuggingFace Inc. team. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

from dataclasses import dataclass, field

from lerobot.common.cameras import CameraConfig

from ..config import RobotConfig


@RobotConfig.register_subclass("koch_screwdriver_follower")
@dataclass
class KochScrewdriverFollowerConfig(RobotConfig):
# Port to connect to the arm
port: str

disable_torque_on_disconnect: bool = True

# `max_relative_target` limits the magnitude of the relative positional target vector for safety purposes.
# Set this to a positive scalar to have the same value for all motors, or a list that is the same length as
# the number of motors in your follower arms.
max_relative_target: int | None = None

# No need to specify cameras here, this is just for typing.
# You can set the cameras in your commands like this:
# python -m lerobot.record \
# --robot.type=koch_screwdriver_follower \
# --robot.port=/dev/servo_5837053138 \
# --robot.cameras="{ screwdriver: {type: opencv, index_or_path: /dev/video0, width: 800, height: 600, fps: 30}, side: {type: opencv, index_or_path: /dev/video2, width: 800, height: 600, fps: 30}}"
cameras: dict[str, CameraConfig] = field(default_factory=dict)

# Set to `True` for backward compatibility with previous policies/dataset
# See the [Hardware API Redesign PR](https://github.com/huggingface/lerobot/pull/777) for more details
use_degrees: bool = False

# Maximum current (raw Dynamixel units) allowed for the screwdriver motor.
# Around 300 corresponds to ~0.8 A on an XL-330 which is plenty for M4 screws
# yet well below the shutdown threshold.
screwdriver_current_limit: int = 300

# Ratio of the above limit at which the software clutch engages.
# When |present_current| >= limit * clutch_ratio the follower will cut the
# screwdriver velocity command to 0 and send haptic feedback to the leader.
clutch_ratio: float = 0.5

# Cool-down duration (seconds) during which any velocity command for the
# screwdriver is forced to zero after the clutch engages. This allows the
# current to fall and prevents repeated brown-outs. Adjust based on your
# control-loop FPS (e.g. 1.0 s ≈ 30–60 frames).
clutch_cooldown_s: float = 1.0
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# Koch Screwdriver

The Koch Screwdriver robot is based on the Koch v1.1, but fitted with a screwdriver instead of a gripper.
See the [Koch v1.1](https://github.com/lerobot/lerobot/blob/main/common/robots/koch_v1.1/koch_v1.1.mdx) documentation for instructions on building the base robot.
The calibration procedure is identical to that of the Koch v1.1.

## How It Works
The screwdriver modification uses the same Dynamixel motor as the Koch v1.1 gripper, but operates it in wheel (also known as velocity) mode.
The movable part of the gripper has been replaced by a magnetic screwdriver holder, and the static part of the gripper has been removed.

## Screwdriver Hardware

The STL files and bill of materials for the screwdriver hardware can be found in the [Koch robotic arm screwdriver](https://github.com/jackvial/koch_robotic_arm_screwdriver) GitHub repository.
Loading