Quantization of Mediapipe models terminated with "Incomplete symbolic shape inference"

[masashikudo910]

When the following Mediapipe models are tried to be quantized, all of them terminate with the same error: "Incomplete symbolic shape inference".

• face_landmark_with_attention_float32.onnx
• selfie_segmentation_float32.onnx
• selfie_segmentation_landscape_float32.onnx
• hair_segmentation_float32.onnx

As an example, the following is the error message on face_landmark_with_attention_float32.onnx. The same message appears on other models, except for the model name.

• Error message

fp_onnx_model = "face_landmark_with_attention_float32.onnx"
quantize_result = quant.quadric_quantize(fp_onnx_model, 100, synthetic_input = True)

2023-08-02 18:48 - INFO - sdk - quantize - Generating synthetic data
2023-08-02 18:48 - INFO - sdk - quantize - Optimized and converted model to opset 14
2023-08-02 18:48 - INFO - sdk - quantize - Saved optimized model to face_landmark_with_attention_float32_float32_opt.onnx
---------------------------------------------------------------------------
Exception                                 Traceback (most recent call last)
Cell In[19], line 6
      3 from sdk_cli.lib.chimera_job import ChimeraJob
      5 fp_onnx_model = "face_landmark_with_attention_float32.onnx"
----> 6 quantize_result = quant.quadric_quantize(fp_onnx_model, 100, synthetic_input = True)

File ~/V2/sdk-cli/sdk_cli/lib/quantize.py:330, in quadric_quantize(floating_point_model_path, num_images, mh, calibration_folder, synthetic_input, exclude_nodes_by_type, exclude_nodes_by_name, asymmetric_activation)
    327     logger.info("Collecting calibration data")
    329 floating_point_model = onnx.load(floating_point_model_path)
--> 330 (optimized_model_path, input_list) = optimize_onnx(floating_point_model_path)
    332 if synthetic_input:
    333     dr = QuadricDataReaderGeneric(input_list)

File ~/V2/sdk-cli/sdk_cli/lib/quantize.py:278, in optimize_onnx(floating_point_model_path, desired_opset)
    275 onnx.save(converted_optimized_model, optimized_model_path)
    276 logger.info(f"Saved optimized model to {optimized_model_path}")
--> 278 quant_pre_process(optimized_model_path, optimized_model_path)
    280 session = onnxruntime.InferenceSession(optimized_model_path)
    281 input_list = get_input_output_info(session)

File ~/V2/tvm/quadric_env/lib/python3.9/site-packages/onnxruntime/quantization/shape_inference.py:70, in quant_pre_process(input_model_path, output_model_path, skip_optimization, skip_onnx_shape, skip_symbolic_shape, auto_merge, int_max, guess_output_rank, verbose, save_as_external_data, all_tensors_to_one_file, external_data_location, external_data_size_threshold)
     68 if not skip_symbolic_shape:
     69     logger.info("Performing symbolic shape inference...")
---> 70     model = SymbolicShapeInference.infer_shapes(
     71         onnx.load(input_model_path),
     72         int_max,
     73         auto_merge,
     74         guess_output_rank,
     75         verbose,
     76     )
     78 if not skip_optimization:
     79     # Use ORT optimizers (native code) to optimize model
     80     if not skip_symbolic_shape:
     81         # Need to save the inferenced model to file so as to run the optimizer

File ~/V2/tvm/quadric_env/lib/python3.9/site-packages/onnxruntime/tools/symbolic_shape_infer.py:2696, in SymbolicShapeInference.infer_shapes(in_mp, int_max, auto_merge, guess_output_rank, verbose)
   2694 if not all_shapes_inferred:
   2695     onnx.save_model(symbolic_shape_inference.out_mp_, "sym_shape_infer_temp.onnx", save_as_external_data=True)
-> 2696     raise Exception("Incomplete symbolic shape inference")
   2697 return symbolic_shape_inference.out_mp_

Exception: Incomplete symbolic shape inference