[help-wanted ]Problems with using GPU

[torsilver]

When running the sample program "Pyramids"
I set Agent's Inference Device as GPU
But MLAgents - Learn seems to call the CPU as well.
I have installed PyTorch1.7.1 and Cuda11.2
Is there a problem with my version of software?

pip list
Package Version

---

absl-py 0.11.0
attrs 20.3.0
cached-property 1.5.2
cachetools 4.2.0
cattrs 1.0.0
certifi 2020.12.5
chardet 4.0.0
cloudpickle 1.6.0
google-auth 1.24.0
google-auth-oauthlib 0.4.2
grpcio 1.34.1
h5py 3.1.0
idna 2.10
importlib-metadata 3.4.0
lxml 4.6.2
Markdown 3.3.3
mlagents 0.23.0
mlagents-envs 0.23.0
numpy 1.18.5
oauthlib 3.1.0
opcua 0.98.12
Pillow 8.1.0
pip 20.3.3
protobuf 3.14.0
pyasn1 0.4.8
pyasn1-modules 0.2.8
pypiwin32 223
python-dateutil 2.8.1
pytz 2020.5
pywin32 300
PyYAML 5.4
requests 2.25.1
requests-oauthlib 1.3.0
rsa 4.7
setuptools 51.3.3
six 1.15.0
tensorboard 2.4.1
tensorboard-plugin-wit 1.8.0
torch 1.7.1+cu110
torchaudio 0.7.2
torchvision 0.8.2+cu110
typing-extensions 3.7.4.3
urllib3 1.26.2
Werkzeug 1.0.1
wheel 0.36.2
zipp 3.4.0

[chriselion]

Hi @torsilver,
The "Inference Device" controls how the Agent performs inference, not training.

Can you run this from the commandline?

python -c 'import torch; print(torch.cuda.is_available())'

If that prints True, ML-Agents should use the GPU for training. If it prints False, there's something wrong with your torch setup, but it's not specific to ML-Agents, so you'll need to look elsewhere to get help debugging it.

[torsilver]

Hi @torsilver,
The "Inference Device" controls how the Agent performs inference, not training.

Can you run this from the commandline?

python -c 'import torch; print(torch.cuda.is_available())'

If that prints True, ML-Agents should use the GPU for training. If it prints False, there's something wrong with your torch setup, but it's not specific to ML-Agents, so you'll need to look elsewhere to get help debugging it.

Hello chriselion
PS C:\Windows\system32> python -c 'import torch; print(torch.cuda.is_available())'
True
I use a laptop with two graphics cards.
The standalone graphics card is GTX1060.
Core display is Intel UHD Graphics 630.
Is it possible to use GPU on ordinary desktop computers?
At present, I want to apply ML-Agents to industrial control of complex systems.
Can I save some training time if I use a GPU?
Do you have any case related to industrial control?

[chriselion]

I use a laptop with two graphics cards.
The standalone graphics card is GTX1060.
Core display is Intel UHD Graphics 630.
Is it possible to use GPU on ordinary desktop computers?

We currently always use device 0. You should be able to override this by setting the CUDA_VISIBLE_DEVICES environment variable before running mlagents-learn - on Mac/Linux, you can do this with CUDA_VISIBLE_DEVICES=1 mlagents-learn ... but I'm not sure if this syntax works in PowerShell. I'm going to add a commandline option to control this better, and add some better logging to make it clear whether torch is using CPU or GPU.

Can I save some training time if I use a GPU?

Unfortunately, I don't have a better answer than "it depends". In general, we haven't seen that reinforcement learning gets much speedup on GPU, but this might change if you use a larger batch size or have visual observations.

Do you have any case related to industrial control?

I don't think anybody has used it for that yet. If you'd like to discuss it more, could you email our team at ml-agents@unity3d.com and we can talk more about it?

[chriselion]

I just merged a change to the master branch that allows setting torch.device from the command line, and also displays the torch device being used for easier debugging. That'll be in the next release, or you can try it out on master now.

[stale]

This issue has been automatically marked as stale because it has not had activity in the last 14 days. It will be closed in the next 14 days if no further activity occurs. Thank you for your contributions.

[stale]

This issue has been automatically closed because it has not had activity in the last 28 days. If this issue is still valid, please ping a maintainer. Thank you for your contributions.

[github-actions]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.