Module loading slow on large systems

[gronke]

I've experienced a multiple second delay when importing iocage on a large system. After figuring out the problem exists when loading modules, I've tried to debug and found a few issues in other places reporting similar occasions. Please excuse my debug print outputs here: https://asciinema.org/a/aISpNg4EIb31BS78uMXFBsxFM

Packages that were found to be slow:

• uuid
• requests

(After resting a self-written implementation of uuid, the next candidate requests shows up).

See also:

• Importing requests is fairly slow psf/requests#3213
• lazily load uuid lib only where needed urllib3/urllib3#1270 (a little step towards better performance)
• https://bugs.python.org/issue11063
• https://bugs.python.org/issue5885

[gronke]

root@ip-172-31-25-0:~ # /usr/bin/time -l python3.6 -m uuid                                                                               
        0.78 real         0.20 user         0.57 sys                
     14588  maximum resident set size                               
         4  average shared memory size                              
         4  average unshared data size                              
       128  average unshared stack size                             
      4379  page reclaims         
         0  page faults           
         0  swaps                 
         8  block input operations                                  
         0  block output operations                                 
         0  messages sent         
         0  messages received     
         0  signals received      
        31  voluntary context switches                              
         9  involuntary context switches                            
root@ip-172-31-25-0:~ # /usr/bin/time -l python3.6 -m hashlib       
        0.03 real         0.02 user         0.01 sys                
     11928  maximum resident set size                               
         5  average shared memory size                              
         5  average unshared data size                              
       160  average unshared stack size                             
      1267  page reclaims         
         0  page faults           
         0  swaps                 
         0  block input operations                                  
         0  block output operations                                 
         0  messages sent         
         0  messages received     
         0  signals received      
         2  voluntary context switches                              
         2  involuntary context switches

[igalic]

We now have a fix in cpython itself: python/cpython#3684

[igalic]

urllib3/urllib3#1270 has been merged!
as soon as they release a new version, we can pin that in our requirements.txt

[gronke]

Will be fixed upstream.