Warning: "mani_skill - WARNING - State dictionary is not consistent, disabling recording of environment states for <TimeLimitWrapper<OrderEnforcing<PickCubeV2Env<PickCube-v2>>>>"

[jstmn]

Hi,

I'm doing Actor.merge in _load_scene() for domain randomization purposes and am getting the warning: "mani_skill - WARNING - State dictionary is not consistent, disabling recording of environment states for <TimeLimitWrapper<OrderEnforcing<PickCubeV2Env>>>"

How can I resolve this? I'd like to have my env states recorded!

Here's my code:

    def _load_scene(self, options: dict):
        ...
        cube_actors = []
        for i in range(self.num_envs):
            builder = self.scene.create_actor_builder()
            builder.add_box_collision(
                half_size=[self.cube_half_size] * 3,
            )
            builder.add_box_visual(
                half_size=[self.cube_half_size] * 3,
                material=sapien.render.RenderMaterial(
                    base_color=np.random.uniform(0, 1, size=4).tolist(),
                ),
            )
            builder.set_scene_idxs([i])
            builder.initial_pose = sapien.Pose(p=[0, 0, self.cube_half_size])
            actor = builder.build_kinematic(name=f"cube_{i}")
            cube_actors.append(actor)

Thanks

• jeremy

[StoneT2000]

When doing heterogeneous simulation (as done here since we have distinct cube actors that are then merged later), the env state will have a key mapping to each cube. If the number of parallel envs is changed then the env state is unusable.

To address this we introduced a state registration system as so: https://maniskill.readthedocs.io/en/latest/user_guide/tutorials/custom_tasks/advanced.html#handling-heterogeneous-simulation-states

This system makes it possible so that even if you collect data with 8192 environments you can replay it even on a different number e.g. 57 environments.

I thought about making this system automatic (e.g. upon actor merge we deregister the individual ones and register the merged one) but decided against it for now.

[jstmn]

OK awesome, that's working.

Just fyi it seems the registration function was renamed to add_to_state_dict_registry()

Complete code for reference:

    def _load_scene(self, options: dict):

        # Create cube actors
        cube_actors = []
        for i in range(self.num_envs):
            builder = self.scene.create_actor_builder()
            builder.add_box_collision(
                half_size=[self.cube_half_size] * 3,
            )
            builder.add_box_visual(
                half_size=[self.cube_half_size] * 3,
                material=sapien.render.RenderMaterial(
                    base_color=[1, 0, 0, 1],
                ),
            )
            builder.set_scene_idxs([i])
            builder.initial_pose = sapien.Pose(p=[0, 0, self.cube_half_size])
            actor = builder.build_kinematic(name=f"cube_{i}")
            self.remove_from_state_dict_registry(actor)
            cube_actors.append(actor)
        self.cube = Actor.merge(cube_actors, name="cube")
        self.add_to_state_dict_registry(self.cube)
        ...

[StoneT2000]

I often write the docs first then write code so probably I forgot to update docs after writing the API. Thanks for the catch.

realized midway programming that register to registry sounded kind of verbose vs add to registry

[jstmn]

Currently the only place that register_to_state_dict_registry() is used is in docs/source/user_guide/tutorials/custom_tasks/advanced.md