Skip to content

simulator should allow pushing objects around with the mouse interactively #1419

@raul-machine-learning

Description

@raul-machine-learning

Not having this is preventing us from testing, debugging and playing with the robots.

According to Mujoco documentation this feature available in their simulator but it is not exposed in lerobot:

A related usability feature is the ability to “reach into” the simulation, push objects around and see how the 
physics respond. The user selects the body to which the external forces and torques will be applied, and sees 
a real-time rendering of the perturbations together with their dynamic consequences. This can be used to debug 
the model visually, to test the response of a feedback controller, or to configure the model into a desired pose.

Also for an awesome OOTB experience it would be great to have a script that loads a pretrained model and makes the interactive simulation just work.

Metadata

Metadata

Assignees

No one assigned

    Labels

    questionRequests for clarification or additional informationsimulationMatters involving system simulation or modeling

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions