Adding Preset Transforms in reference scripts

[datumbox]

Updating the following reference scripts:

• Classification
• Object Detection
• Segmentation
• Video Classification

The Similarity reference script was skipped because it's not a real recipe. A reference implementation for it can be seen at 71b7091.

[fmassa]

The proposed approach looks great to me, good to go for the other tasks as well!

[codecov]

Codecov Report

Merging #3317 (ba326de) into master (7621a8e) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #3317   +/-   ##
=======================================
  Coverage   73.93%   73.93%           
=======================================
  Files         104      104           
  Lines        9594     9594           
  Branches     1531     1531           
=======================================
  Hits         7093     7093           
  Misses       2024     2024           
  Partials      477      477           

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7621a8e...ba326de. Read the comment docs.

[fmassa]

Changes for segmentation and video classification looks good to me as well!

[zhangguanheng66]

Usually, for text domain, we will need to download the transform, for example sentencepiece model, or a vocabulary saved in text file.

[datumbox]

I'm writing here what we discussed on the call.

It seems that supporting your case is possible by using PyTorch Hub's load_state_dict_from_url() method and then passing the result to your code. This is very common pattern in TorchVision, used mainly for pre-trained models. Example:

vision/torchvision/models/mobilenetv3.py

Lines 244 to 249 in 97885cb

	model = MobileNetV3(inverted_residual_setting, last_channel, **kwargs)
	if pretrained:
	if model_urls.get(arch, None) is None:
	raise ValueError("No checkpoint is available for model type {}".format(arch))
	state_dict = load_state_dict_from_url(model_urls[arch], progress=progress)
	model.load_state_dict(state_dict)

[netw0rkf10w]

@datumbox @fmassa Could you please tell me what was the original motivation of this PR? I couldn't find any information on Preset Transforms. Thanks a lot!

[datumbox]

@netw0rkf10w Providing the training and inference transforms of each model/pipeline in an organized way so that people can reproduce the models. Here is an example of complex presets: 1 and 2.

[netw0rkf10w]

Thanks, @datumbox, for your reply! Is there a discussion thread (or a GitHub issue) on the topic that I can read, or was it internal?

[datumbox]

Unfortunately many of these discussions happened outside of Github. This is quite problematic and we want to change it because in circumstances like this, it's hard to give information to people and does not help with the transparency... So, apologies for not being able to point you to a public thread... To solve the situation, below I provide a summary of what motivated the change. Let me know if you need more info:

Preset preprocessing transforms are those transformations applied to the data before feeding them to an ML model. They are typically separated into two categories: those applied during training and those during inference. Examples of such transforms include the Data Augmentation techniques, the Normalization/Scaling methods and other adhoc transformations applied to the data as a preliminary step (binary to bitmap conversion for Vision, Fast Fourier Transforms for Audio, Tokenization for Text etc).

The preset transforms are a crucial part of the model and having access to them is necessary to understand how a model was created and how to use it. Disclosing which training transforms were used is an important part for reproducibility and crucial to understand the assumptions and properties a specific model. The latter is particularly true while using transfer learning and porting a model from one domain to another (for example the Zoom transform can be used for augmentation in ImageNet Classification but not for Cancer Detection). Similarly having access to the transforms used during inference is critical because without them one can’t use the model.

This is the reason we decided to bring these Preset transforms as close to the training references as possible. By putting them together, the users are able to reproduce the training, adjust the scripts to meet their needs, do transfer learning etc. Hope that makes sense.

[netw0rkf10w]

@datumbox That totally makes sense! Thank you very much for your detailed explanation!