feat: add best sparse model flag

[luishresende]

Add --use-best-sparse-model flag in ns-process-data

Context 📝

When using the exhaustive method to estimate poses, COLMAP sometimes generates multiple sparse models in colmap/sparse/. In these cases, model 0 is not always the best and can have far fewer registered images than the other models. Currently, the pipeline refines intrinsics and saves the transforms.json always based on model 0.

Problem ⚠️

This behavior can lead to suboptimal results when model 0 has fewer registered images than other sparse reconstructions. While it is technically possible for users to select the best model manually by running ns-process-data twice with additional parameters, this process is cumbersome and error-prone, especially for new users.

Solution 💡

This PR introduces a flag (--use-best-sparse-model) that allows users to choose whether to use the best sparse model (i.e., the one with the most registered images) for both refining intrinsics and saving the transforms.json. The flag is enabled by default:

use_best_sparse_model: bool = True
"""If True, use the best sparse model to refine intrinsics and save the camera transformations.
Only works with colmap sfm_tool"""

To support this, the PR adds a helper class called BestSparseModel, which:

• Automatically finds the sparse model in colmap/sparse/ with the most registered images.
• Requires the colmap_dir path to locate the sparse models.
• Caches the result to avoid redundant computations, since the best model is accessed multiple times in the pipeline.
• Provides both the model name and full path via get_model() and get_model_path(), reducing code repetition.

Usage in the pipeline 🔧

1. In process_data/colmap_utils.py, when refine_intrinsics is enabled:

    if refine_intrinsics:
        sparse_model = "0" if not use_best_sparse_model else BestSparseModel.get_model(colmap_dir)

        with status(msg="[bold yellow]Refine intrinsics...", spinner="dqpb", verbose=verbose):
            bundle_adjuster_cmd = [
                f"{colmap_cmd} bundle_adjuster",
                f"--input_path {sparse_dir}/{sparse_model}",
                f"--output_path {sparse_dir}/{sparse_model}",
                "--BundleAdjustment.refine_principal_point 1",
            ]
            run_command(" ".join(bundle_adjuster_cmd), verbose=verbose)
        CONSOLE.log("[bold green]:tada: Done refining intrinsics.")

This ensures that the best sparse model is selected for intrinsics refinement.

2. In process_data/colmap_converter_to_nerfstudio_dataset.py, the class ColmapConverterToNerfstudioDataset has the property absolute_colmap_model_path modified:

    @property
    def absolute_colmap_model_path(self) -> Path:
        if not self.skip_colmap and self.use_best_sparse_model:
            return colmap_utils.BestSparseModel.get_model_path(self.absolute_colmap_path)
        return self.output_dir / self.colmap_model_path

This ensures that whenever colmap_model_path is requested, it returns the path of the best model when the flag is enabled. For example, in the function _save_transforms:

        if (self.absolute_colmap_model_path / "cameras.bin").exists():
            with CONSOLE.status("[bold yellow]Saving results to transforms.json", spinner="balloon"):
                num_matched_frames = colmap_utils.colmap_to_json(
                    recon_dir=self.absolute_colmap_model_path,
                    output_dir=self.output_dir,
                    image_id_to_depth_path=image_id_to_depth_path,
                    camera_mask_path=camera_mask_path,
                    image_rename_map=image_rename_map,
                    use_single_camera_mode=self.use_single_camera_mode,
                )

This ensures that the transforms.json is generated based on the best sparse model.

Expected Impact 🌟

With the --use-best-sparse-model flag enabled by default, the pipeline will now:

• Automatically select the sparse model with the most registered images for intrinsics refinement.
• Generate transforms.json based on the best available reconstruction, instead of always using model 0.
• Improve reliability and accuracy when COLMAP produces multiple sparse models, especially in challenging datasets.
• Maintain backward compatibility, as the behavior can be disabled by setting the flag to False ✅.

[ichsan2895]

I hope nerfstudio is not dead, so any developers can take a look on this interesting feature

[ncan33]

This is an excellent feature. I've encountered this issue multiple times, especially when working with object-centric images. When there are few features, model 0 is rarely the best one. Hopefully the maintainers merge this.