[vizualizer] for LeRobodDataset V2 #576

mishig25 · 2024-12-13T16:10:34Z

Description

https://huggingface.co/spaces/lerobot/visualize_dataset is already using this branch

As we discussed offline, this PR updates the visualizer to use v2 datasets and merge all the visualizer changes from https://huggingface.co/spaces/lerobot/visualize_dataset fork into lerobot repository

Usage

# start a server with homepage will fetch datasets hosted remotely on hf.co/datasets
python lerobot/scripts/visualize_dataset_html.py

# start a server with specific dataset (by default it will download parquet & video files locally AND start the visualizer server)
python lerobot/scripts/visualize_dataset_html.py --repo-id lerobot/aloha_static_ziploc_slide

# start a server with specific dataset without downloading parquet & video files locally (instead the visualizer server will fetch the dataset hosted remotely on hf.co/dataasets) `-load-from-hf-hub 1` arg
python lerobot/scripts/visualize_dataset_html.py --repo-id lerobot/aloha_static_ziploc_slide --load-from-hf-hub 1

besides the v2 dataset, further improvements were added to vizualizer

using the specified motor names
see example here

refactored the code in a way that would let us easily add more columns to the data table
After merging this PR, I will add [viz tool] add policy pred column #529

This reverts commit 27bf982.

mishig25 · 2024-12-16T11:17:29Z

cc: @aliberts @Cadene ready to review

mishig25 · 2024-12-16T11:21:38Z

tests/test_visualize_dataset_html.py

@@ -1,30 +0,0 @@
-#!/usr/bin/env python


this test file was removed since the entire functionality was removed. I added changes from #387 to this PR

aliberts

Awesome, thank you @mishig25!
Left a non-blocking comment, feel free to follow or merge directly

aliberts · 2024-12-20T09:33:03Z

lerobot/common/datasets/utils.py

@@ -20,7 +20,8 @@
 from itertools import accumulate
 from pathlib import Path
 from pprint import pformat
-from typing import Any
+from types import SimpleNamespace
+from typing import Any, Dict, Iterator


Nit: since we're using python >=3.10 we don't need typing.Dict and we can type-hint directly with dict.
Similarly, typing.Iterator is deprecated and can be replaced by collections.abc.Iterable

handled in c425bd8

mishig25 · 2024-12-20T13:03:59Z

I pushed c425bd8 that makes it use the modern types.

@aliberts please feel free to merge since I don't have premissions to merge it even though the PR was approved

aliberts · 2024-12-20T15:26:20Z

Sure thing, thanks!

* feat: enable to use multiple rgb encoders per camera in diffusion policy (huggingface#484) Co-authored-by: Alexander Soare <[email protected]> * Fix config file (huggingface#495) * fix: broken images and a few minor typos in README (huggingface#499) Signed-off-by: ivelin <[email protected]> * Add support for Windows (huggingface#494) * bug causes error uploading to huggingface, unicode issue on windows. (huggingface#450) * Add distinction between two unallowed cases in name check "eval_" (huggingface#489) * Rename deprecated argument (temporal_ensemble_momentum) (huggingface#490) * Dataset v2.0 (huggingface#461) Co-authored-by: Remi <[email protected]> * Refactor OpenX (huggingface#505) * Fix missing local_files_only in record/replay (huggingface#540) Co-authored-by: Simon Alibert <[email protected]> * Control simulated robot with real leader (huggingface#514) Co-authored-by: Remi <[email protected]> * Update 7_get_started_with_real_robot.md (huggingface#559) * LerobotDataset pushable to HF from any folder (huggingface#563) * Fix example 6 (huggingface#572) * fixing typo from 'teloperation' to 'teleoperation' (huggingface#566) * [vizualizer] for LeRobodDataset V2 (huggingface#576) * Fix broken `create_lerobot_dataset_card` (huggingface#590) * feat(act): support training end of episode token to ACT model * changes * feat(arx): add arx arm (#2) * feat(arx): support arx arm * changes * changes * changes * changes * pass pipes explicitly * changes * us ndarray over a pipe * changes * changes * replay basically works * patch arx sdk * changes * support cameras in arx5 * rename to arx5 * kind of works * changes * changes * changes * various changes * changes * revert a few changes * changes * changes * changes * changes * changes * changes * changes * changes * changes * remove TODO * allow multiple tasks --------- Signed-off-by: ivelin <[email protected]> Co-authored-by: Hirokazu Ishida <[email protected]> Co-authored-by: Alexander Soare <[email protected]> Co-authored-by: Arsen Ohanyan <[email protected]> Co-authored-by: Ivelin Ivanov <[email protected]> Co-authored-by: Daniel Ritchie <[email protected]> Co-authored-by: resolver101757 <[email protected]> Co-authored-by: Jannik Grothusen <[email protected]> Co-authored-by: KasparSLT <[email protected]> Co-authored-by: Simon Alibert <[email protected]> Co-authored-by: Remi <[email protected]> Co-authored-by: Michel Aractingi <[email protected]> Co-authored-by: Simon Alibert <[email protected]> Co-authored-by: berjaoui <[email protected]> Co-authored-by: Claudio Coppola <[email protected]> Co-authored-by: s1lent4gnt <[email protected]> Co-authored-by: Mishig <[email protected]> Co-authored-by: Eugene Mironov <[email protected]>

* feat: enable to use multiple rgb encoders per camera in diffusion policy (huggingface#484) Co-authored-by: Alexander Soare <[email protected]> * Fix config file (huggingface#495) * fix: broken images and a few minor typos in README (huggingface#499) Signed-off-by: ivelin <[email protected]> * Add support for Windows (huggingface#494) * bug causes error uploading to huggingface, unicode issue on windows. (huggingface#450) * Add distinction between two unallowed cases in name check "eval_" (huggingface#489) * Rename deprecated argument (temporal_ensemble_momentum) (huggingface#490) * Dataset v2.0 (huggingface#461) Co-authored-by: Remi <[email protected]> * Refactor OpenX (huggingface#505) * Fix missing local_files_only in record/replay (huggingface#540) Co-authored-by: Simon Alibert <[email protected]> * Control simulated robot with real leader (huggingface#514) Co-authored-by: Remi <[email protected]> * Update 7_get_started_with_real_robot.md (huggingface#559) * LerobotDataset pushable to HF from any folder (huggingface#563) * Fix example 6 (huggingface#572) * fixing typo from 'teloperation' to 'teleoperation' (huggingface#566) * [vizualizer] for LeRobodDataset V2 (huggingface#576) * Fix broken `create_lerobot_dataset_card` (huggingface#590) * Update README.md (huggingface#612) * Fix Quality workflow (huggingface#622) * fix(docs): typos in benchmark readme.md (huggingface#614) Co-authored-by: Simon Alibert <[email protected]> * fix(visualise): use correct language description for each episode id (huggingface#604) Co-authored-by: Simon Alibert <[email protected]> * typo fix: batch_convert_dataset_v1_to_v2.py (huggingface#615) Co-authored-by: Simon Alibert <[email protected]> * [viz] Fixes & updates to html visualizer (huggingface#617) * fixes to SO-100 readme (huggingface#600) Co-authored-by: Philip Fung <no@one> Co-authored-by: Simon Alibert <[email protected]> --------- Signed-off-by: ivelin <[email protected]> Co-authored-by: Hirokazu Ishida <[email protected]> Co-authored-by: Alexander Soare <[email protected]> Co-authored-by: Arsen Ohanyan <[email protected]> Co-authored-by: Ivelin Ivanov <[email protected]> Co-authored-by: Daniel Ritchie <[email protected]> Co-authored-by: resolver101757 <[email protected]> Co-authored-by: Jannik Grothusen <[email protected]> Co-authored-by: KasparSLT <[email protected]> Co-authored-by: Simon Alibert <[email protected]> Co-authored-by: Remi <[email protected]> Co-authored-by: Michel Aractingi <[email protected]> Co-authored-by: Simon Alibert <[email protected]> Co-authored-by: berjaoui <[email protected]> Co-authored-by: Claudio Coppola <[email protected]> Co-authored-by: s1lent4gnt <[email protected]> Co-authored-by: Mishig <[email protected]> Co-authored-by: Eugene Mironov <[email protected]> Co-authored-by: CharlesCNorton <[email protected]> Co-authored-by: Philip Fung <[email protected]> Co-authored-by: Philip Fung <no@one>

[Fix] Move back to manual calibration (#488) feat: enable to use multiple rgb encoders per camera in diffusion policy (#484) Co-authored-by: Alexander Soare <[email protected]> Fix config file (#495) fix: broken images and a few minor typos in README (#499) Signed-off-by: ivelin <[email protected]> Add support for Windows (#494) bug causes error uploading to huggingface, unicode issue on windows. (#450) Add distinction between two unallowed cases in name check "eval_" (#489) WIP Fix autocalib moss (#486) [Fix] Move back to manual calibration (#488) feat: enable to use multiple rgb encoders per camera in diffusion policy (#484) Co-authored-by: Alexander Soare <[email protected]> Fix config file (#495) fix: broken images and a few minor typos in README (#499) Signed-off-by: ivelin <[email protected]> Add support for Windows (#494) bug causes error uploading to huggingface, unicode issue on windows. (#450) Add distinction between two unallowed cases in name check "eval_" (#489) Rename deprecated argument (temporal_ensemble_momentum) (#490) Dataset v2.0 (#461) Co-authored-by: Remi <[email protected]> Refactor OpenX (#505) Fix missing local_files_only in record/replay (#540) Co-authored-by: Simon Alibert <[email protected]> Control simulated robot with real leader (#514) Co-authored-by: Remi <[email protected]> Update 7_get_started_with_real_robot.md (#559) LerobotDataset pushable to HF from any folder (#563) Fix example 6 (#572) fixing typo from 'teloperation' to 'teleoperation' (#566) [vizualizer] for LeRobodDataset V2 (#576) Fix broken `create_lerobot_dataset_card` (#590) Update README.md (#612) Add draccus, create MainConfig WIP refactor train.py and ACT Add policies training presets Update diffusion policy Add pusht and xarm env configs Update tdmpc Update vqbet Fix poetry relax Add feature types to envs Add EvalPipelineConfig, parse features from envs Add custom parser Update pretrained loading mechanisms Add dependency fixes & lock update Fix pretrained_path Refactor envs, remove RealEnv Fix typo Enable end-to-end tests Fix Makefile Log eval config Fix end-to-end tests Fix Quality workflow (#622) Remove amp & add resume test Speed-up tests Fix poetry relax Remove config yaml for robot devices (#594) Co-authored-by: Simon Alibert <[email protected]> fix(docs): typos in benchmark readme.md (#614) Co-authored-by: Simon Alibert <[email protected]> fix(visualise): use correct language description for each episode id (#604) Co-authored-by: Simon Alibert <[email protected]> typo fix: batch_convert_dataset_v1_to_v2.py (#615) Co-authored-by: Simon Alibert <[email protected]> [viz] Fixes & updates to html visualizer (#617) Fix logger Remove hydra-core Add aggregate_stats Add estimate_num_samples for images, Add test image Remove NoneSchedulerConfig Add push_pretrained Remove eval.episode_length Fix wandb_video Fix typo Add features back into policy configs (#643) fixes to SO-100 readme (#600) Co-authored-by: Philip Fung <no@one> Co-authored-by: Simon Alibert <[email protected]> Fix for the issue #638 (#639) Fix env_to_policy_features call Fix wandb init remove omegaconf Add branch arg Move deprecated Move training config Remove pathable_args Implement custom HubMixin Fixes Implement PreTrainedPolicy base class Add HubMixin to TrainPipelineConfig Udpate example 2 & 3 Update push_pretrained Bump`rerun-sdk` dependency to `0.21.0` (#618) Co-authored-by: Simon Alibert <[email protected]> Fix config_class Fix from_pretrained kwargs Remove policy_protocol Camelize PretrainedConfig Additional fix while retraining policies (#629) Co-authored-by: Simon Alibert <[email protected]> Actually reactivate tdmpc online test Update example 4 Remove advanced example 1 Remove example 5 Move example 6 to advanced Use HubMixin.save_pretrained Enable config_path to be a repo_id Dry has_method Update example 4 Update README Cleanup pyproject.toml Update eval docstring Update README Clean example 4 Update README Make 'last' checkpoint symlink relative Fix cluster image (#653) Simplify example 4 fix stats per episodes and aggregate stats and casting to tensor

mishig25 added 30 commits December 3, 2024 11:21

add hmtl changes

04656bd

use .meta properties

0a9d8d7

correct html template

283a675

fix dim keys

27bf982

dataset["features"]

cc474f0

Revert "fix dim keys"

b360f22

This reverts commit 27bf982.

fix meta keys

ae2c617

format

8598d14

fix padding

5af2871

fix filenames

d4bc181

fix meta version

40164d1

debug

3d5ed47

fix cache

545b1bb

fix cache more

51c6cdf

debug

4032783

use pandas directly

4d8d44c

less diff with main

39d5aa4

use namespace

6a432dd

doc

ca5a721

more dot notation

d51cf86

more dot notation

09eac27

correct task index

03c35ee

pass column motor names

4ac7738

generalize columns in html

9738dba

fix field motors names

7e82572

refactor columns logic

c427d71

Merge branch 'main' into v2_dataset_viz

05e778a

rm unused time_param = all_params["t"]

8cfac30

rm dev

84a0baa

ruff check --fix

48c2254

mishig25 added 5 commits December 13, 2024 17:24

use row labels

5e14006

fix wrap

832ae0c

rm unneeded test

42812df

rm debug

cdd032c

better v2 error msg

de382d6

mishig25 marked this pull request as ready for review December 16, 2024 11:16

mishig25 commented Dec 16, 2024

View reviewed changes

fix image_keys

d605f0c

aliberts approved these changes Dec 20, 2024

View reviewed changes

use python3.10> types

c425bd8

aliberts merged commit 73b64c3 into huggingface:main Dec 20, 2024
5 checks passed

chrisheninger pushed a commit to chrisheninger/lerobot that referenced this pull request Jan 26, 2025

[vizualizer] for LeRobodDataset V2 (huggingface#576)

c4f9096

michel-aractingi pushed a commit that referenced this pull request Feb 3, 2025

[vizualizer] for LeRobodDataset V2 (#576)

0a4e9e2

menhguin pushed a commit to menhguin/lerobot that referenced this pull request Feb 9, 2025

[vizualizer] for LeRobodDataset V2 (huggingface#576)

3bffc66

JIy3AHKO pushed a commit to vertix/lerobot that referenced this pull request Feb 27, 2025

[vizualizer] for LeRobodDataset V2 (huggingface#576)

7ef3192

Kalcy-U referenced this pull request in Kalcy-U/lerobot May 13, 2025

[vizualizer] for LeRobodDataset V2 (#576)

933bb55

ZoreAnuj pushed a commit to luckyrobots/lerobot that referenced this pull request Jul 29, 2025

[vizualizer] for LeRobodDataset V2 (huggingface#576)

d23b783

Ricci084 pushed a commit to JeffWang987/lerobot that referenced this pull request Sep 5, 2025

[vizualizer] for LeRobodDataset V2 (huggingface#576)

534b0b9

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[vizualizer] for LeRobodDataset V2 #576

[vizualizer] for LeRobodDataset V2 #576

Uh oh!

mishig25 commented Dec 13, 2024 •

edited

Loading

Uh oh!

mishig25 commented Dec 16, 2024

Uh oh!

mishig25 Dec 16, 2024 •

edited

Loading

Uh oh!

aliberts left a comment

Uh oh!

aliberts Dec 20, 2024 •

edited

Loading

Uh oh!

mishig25 Dec 20, 2024

Uh oh!

mishig25 commented Dec 20, 2024

Uh oh!

aliberts commented Dec 20, 2024

Uh oh!

Uh oh!

Uh oh!

[vizualizer] for LeRobodDataset V2 #576

[vizualizer] for LeRobodDataset V2 #576

Uh oh!

Conversation

mishig25 commented Dec 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Usage

besides the v2 dataset, further improvements were added to vizualizer

Uh oh!

mishig25 commented Dec 16, 2024

Uh oh!

mishig25 Dec 16, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

aliberts left a comment

Choose a reason for hiding this comment

Uh oh!

aliberts Dec 20, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mishig25 Dec 20, 2024

Choose a reason for hiding this comment

Uh oh!

mishig25 commented Dec 20, 2024

Uh oh!

aliberts commented Dec 20, 2024

Uh oh!

Uh oh!

Uh oh!

mishig25 commented Dec 13, 2024 •

edited

Loading

mishig25 Dec 16, 2024 •

edited

Loading

aliberts Dec 20, 2024 •

edited

Loading