I'm now at 41.2 it/s, up from 39.2 #7860
Replies: 11 comments 21 replies
|
Actually closer to 41.4 100%|███████████| 20/20 [00:00<00:00, 41.53it/s] |
|
I have many questions about this topic. Here goes the questions: What model are you using for testing? In my case testing:
On the one hand, there is some volatility in terms of speed, on the other hand, the way of representation is different? I have gone through the various steps you mention and have not gone any further. Any suggestions? |
|
Lately I have seen so many people with performance problems of the 30xx or 40xx cards. They are slower than my card (2080). I read a comment somewhere mentioning that an older xformers version might solve it, but its a rumour I can't test it. Something is going on |
|
Maybe part of my problem comes from OS settings... My previous GPU was a 2060, its the same OS, have no idea how to check it. |
|
Is there a solution for Windows? I tried many things, most of them have no effect, while some make it worse. |
|
Even better, with --opt-channelslast I get 42.1 it/s |
|
Sorry but which .py file are you adding the original change to? |
What "original change"? I'm sure I've mentioned several things. |
|
Has no effect on my setup. |
|
I meant which 'main py file'? |
|
Where did you add that |
|
this made my 4090 slower |
|
Has no effect on my setup. Window 10, 4070 Ti. |
|
@aifartist Can you tell me which py file to modify torch.compile(mode='max-autotune') and batchsize=4 in? thank you very much |
|
modules/sd_hijack.py. However, don't expect it to actually work. When PYT 2.0 came out something got broken because I use to have been successful using it. And I do not have time to debug that now. Vlado's fork has the modification to sd_hijack.py but, as I just said, unless you are a hard core hacker that can try a bunch of old versions of pytorch to find one where it barely worked you WILL NOT be able to use this. |
|
thank you very much，I'll do some research |
|
Thank you, this increased my speed as well with a 4080 |
|
@CHollman82 This is the numbers I get with torch.compile() but I have to make fixes to the pytorch code to get it to work. |
|
Cant wait to get a 4090 too. I probably wont reach these top scores because my CPU is a bit older (Ryzen 3600X) If TensorRT and distilled diffusion models will work together... this is going to get crazy. Real time prompt rendering and living in an animation wonderland |
|
How did you get this crazy speed? My 4090 can only get a maximum speed of 23it/s under the full default setting in webui. And I have replaced the latest cudnn file and used the latest xformers. |
|
If you have a slow CPU and a uber fast GPU you can easily see only 23 it/s. I assume you aren't using a very long prompt when just benchmarking. |
|
24.) Automatic1111 Web UI - PC - Free |
|
Pls add more info, what exactly you did, which OS, video card, what were the gains etc. AFAIK, torch 2.0 does not improve performance. |
windows 10 i felt increase but maybe not much . i think new cards getting more |
|
I have 4070 Ti. The only thing that increased performance by +40% for me was replacing CUDA dlls, from 5it/sec to 7, not a lot. |
|
Which exact versions of CUDA files you used? I was looking into this thing and was also looking to replace CUDA files and see if it made any difference. |
|
I get 1.51 it/s with my 4070. Ha. Even with your addition to main.py and the command args. |
|
Try SD.Next with Stability Matrix. I am getting 25 it/sec on 4070 Ti. Installs in a few clicks. No parameter tweaking. |
Cool - looking into that now. Just spent two days (after working fine) fighting with Automatic1111 - cuda memory errors instantly. Torch not compiled with cuda/gpu. Nonetype errors. Mat1 and Mat2 errors. Hahaha! |
|
Is anyone running SD.NEXT with a 4070 Ti 12GB and cu121? I checked the published benchmarks and couldn't find any that matched my setup. My benchmark below timestamp 2024-01-14 04:33:03.298575 I feel like I should be getting better speeds but can't seem to find the right settings. What I have now at least is stable. Trying some changes leads to crash. |
|
I only got 12-14s/it Running SD A1111 (v1.10.1) on RTX A6000. Loaded XL checkpoint (realvisxlv40_v20Bakedvae), ControlNet XL ip-(adaptor-plus-sdxl-vit-h) to process 1080 x 1920 images. Using --xformer and --medvram, live preview disabled. Batch sizee=4, Batch count=1 (but I do enabled "Batch Loopback"-since my batch count=1, it shouldn't do anything?) Not using Loopback in Script. GPU temp is 81 Celsius. Thanks! ++++++++Run Report+++++++ To create a public link, set Inpaint batch is enabled. 3 masks found. |
Uh oh!
There was an error while loading. Please reload this page.
In addition to #6954, which tripled my performance, I just found that adding:
torch.backends.cudnn.benchmark = Trueto the main py file in A1111 got me up to 41.2 it/s.
I had seen this option before but hadn't noticed any speedup, but I was less savvy at that time.
A question for our NN experts here: Do 'input sizes' change 'often' in any of the various and many A1111 functional pieces?
If that does occur there can be negative impacts from using this option depending on how often the sizes change.
FYI, Today I helped another windows user go from 10 it/s to about 30 it/s. When I upgraded the cuDNN dll's to v8.7 it only went to 16 it/s. But then I discovered that bat file was setting --precision=full and when that was removed it then went to 30.
Is precision=full needed for anything other than training or to prevent black images when using sd v2?
All reactions