Pre-trained detection models for PyTorch Hub

[chenliu0831]

🚀 Feature

First of all, thanks for the amazing work.

I noticed currently there's no pre-trained detection models exposed via hub_conf.py. It looks like the building blocks are already there. Is there plan to support it?

Thanks

Motivation

Increase coverage of supported vision tasks via Torch Hub.

Pitch

• entrypoints for detection models are added to hub_conf.py
• some helpers e.g. for bbox parsing could also be added there

[fmassa]

Hi,

Adding the detection models in hub_conf.py wouldn't be enough in general, because those models require custom C++ / CUDA ops which are only present in torchvision.
Given that TorchHub doesn't compile the repos it clones, we wouldn't have a full-fleshed detection model with the current setup for TorchHub.

ccing @ailzhang to see if there are other options.

[chenliu0831]

Gotcha. Related, I think Fairseq actually is doing some Cython compilation inside their hubconf.py. Might be an option if the custom op compilation is fast.

[fmassa]

@chenliu0831 understood, that could be a solution for it, although I'm not sure if we should be doing any compilation in the torchhub setup, as it would take a long time to run.

[ailzhang]

@fmassa IMHO it's worth adding to make the detection models more accessible. Users will build for the first time, but as we cache the build, they should be good until they force a rebuild.

[fmassa]

We could try adding support for it, although it will require users to have a C++ compiler + nvcc (if using GPUs), which might not always be the case, so we would need to give good error messages in those cases.

[chenliu0831]

Awesome, I'll take a shot.

[AdeelH]

@chenliu0831, any update on this?

[AdeelH]

@fmassa, I tried adding the detection models to hubconf.py (AdeelH@2b35026) and importing them via torch hub. I found that it mostly works.

Note that I did NOT build either pytorch or torchvision from source. I installed them using

pip3 install torch==1.10.2+cu113 torchvision==0.11.3+cu113 torchaudio==0.10.2+cu113 -f https://download.pytorch.org/whl/cu113/torch_stable.html

This is the code I used to test the imports:

import torch
import torchvision

model = torch.hub.load( 
	'/home/adeel/vision',
	'ssd300_vgg16',
	source='local',
	num_classes=2,
	force_reload=True,
	pretrained=False,
	pretrained_backbone=False).eval()

model(torch.zeros(1, 3, 300, 300))

Observations

• The snippet above fails if I make the hubconf.py changes over the main branch.
• The snippet above succeeds if I make the hubconf.py changes over the 0.11 release, presumably because the version matches my torchvision installation's version.
• The snippet above fails if I remove import torchvision (line 2).

In each failure case, I get this error:

RuntimeError: Couldn't load custom C++ ops. This can happen if your PyTorch and torchvision versions are incompatible, or if you had errors while compiling torchvision from source. For further information on the compatible versions, check https://github.com/pytorch/vision#installation for the compatibility matrix. Please check your PyTorch version with torch.__version__ and your torchvision version with torchvision.__version__ and verify if they are compatible, and if not please reinstall torchvision so that it matches your PyTorch install.

---

What do you make of this? Is it worth adding the detection models to hubconf.py?