Introduce Preprocessing for Optimized Quantization in quantize-ort.py

[Tim-Siu]

Issues

Resolve #239

Running the quantization script quantize-ort.py cannot reproduce the quantized model in the repo. The current script will produce int8 quantized ppresnet50 at a size of over 120 MB, which is significantly different from the existing quantized models in the repo at the size of ~26 MB. After some investigation, I think the reason might be that preprocessing is missing. The ONNX documentation seems to suggest preprocessing is highly encouraged.

Left: Computation graph of already quantized models in the repo or models quantized by the updated script.
Right: Computation graph of the model quantized by the original script.

We can see that the current script will result in a model with an unoptimized computation graph and redundant computation nodes.

Key Changes

• A preprocessing step is added in quantize-ort.py. Optimization is automatically carried out by the quant_pre_process method.

Expected Benefits

• Running the updated script should allow us to reproduce the quantized models already in the repo.

[Tim-Siu]

By the way, I am very interested in working on the "Quantized models for OpenCV Model Zoo" project as part of Google Summer of Code. I am looking forward to any opportunities to contribute to opencv_zoo :)

[fengyuentau]

Thank you for the contribution!

[fengyuentau]

By the way, I am very interested in working on the "Quantized models for OpenCV Model Zoo" project as part of Google Summer of Code. I am looking forward to any opportunities to contribute to opencv_zoo :)

We will see whether we have such project in this year's GSoC. In the meantime, you are welcome to apply for other projects.

[fengyuentau]

Also attaching this page for reference if cli is preferred: https://github.com/microsoft/onnxruntime-inference-examples/blob/main/quantization/image_classification/cpu/ReadMe.md