0.29.1: RecursionError: maximum recursion depth exceeded while calling a Python object

[yuezhongtao]

eventlet version:
Name : python3-eventlet
Version : 0.29.1
Release : 412

os:
Linux E2E0A942-2AB1-8546-AB44-B994FF1EB2B2 4.18.0-147.5.1.0.h276.eulerosv2r9.x86_64 #1 SMP Thu Nov 26 17:26:42 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

python: Python 3.7.4

crash error trace log：

2020-12-11T14:56:40.665+08:00 localhost neutron-server ERROR [pid:363233] [GreenThread-1835][tid:22489070077920] [catch_errors.py:44 __call__] An error occurred during processing the request: GET /v2.0/networks?name=external_om HTTP/1.0^M
Accept: application/json^M
Accept-Encoding: gzip, deflate^M
Connection: close^M
Content-Type: application/json^M
Host: network.localdomain.com:8020^M
User-Agent: python-requests/2.22.0^M
X-Auth-Token: *****: RecursionError: maximum recursion depth exceeded while calling a Python object
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors Traceback (most recent call last):
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/oslo_middleware/catch_errors.py", line 40, in __call__
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     response = req.get_response(self.application)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/webob/request.py", line 1314, in send
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     application, catch_exc_info=False)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/webob/request.py", line 1278, in call_application
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     app_iter = application(self.environ, start_response)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/webob/dec.py", line 129, in __call__
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     resp = self.call_func(req, *args, **kw)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/webob/dec.py", line 193, in call_func
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     return self.func(req, *args, **kwargs)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/osprofiler/web.py", line 122, in __call__
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     return request.get_response(self.application)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/webob/request.py", line 1314, in send
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     application, catch_exc_info=False)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/webob/request.py", line 1278, in call_application
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     app_iter = application(self.environ, start_response)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/webob/dec.py", line 129, in __call__
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     resp = self.call_func(req, *args, **kw)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/webob/dec.py", line 193, in call_func
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     return self.func(req, *args, **kwargs)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/keystonemiddleware/auth_token/__init__.py", line 338, in __call__
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     response = self.process_request(req)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/keystonemiddleware/auth_token/__init__.py", line 660, in process_request
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     resp = super(AuthProtocol, self).process_request(request)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/keystonemiddleware/auth_token/__init__.py", line 411, in process_request
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     allow_expired=allow_expired)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/keystonemiddleware/auth_token/__init__.py", line 446, in _do_fetch_token
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     data = self.fetch_token(token, **kwargs)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/keystonemiddleware/auth_token/__init__.py", line 755, in fetch_token
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     allow_expired=allow_expired)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/keystonemiddleware/auth_token/_identity.py", line 180, in verify_token
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     allow_expired=allow_expired)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/keystonemiddleware/auth_token/_identity.py", line 75, in verify_token
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     access_rules_support=ACCESS_RULES_SUPPORT)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/keystoneclient/v3/tokens.py", line 120, in validate
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     access_rules_support=access_rules_support)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/keystoneclient/v3/tokens.py", line 93, in get_token_data
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     resp, body = self._client.get(url, headers=headers)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/keystoneclient/v3/tokens.py", line 93, in get_token_data
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     resp, body = self._client.get(url, headers=headers)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/keystoneauth1/adapter.py", line 386, in get
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     return self.request(url, 'GET', **kwargs)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/keystoneauth1/adapter.py", line 545, in request
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     resp = super(LegacyJsonAdapter, self).request(*args, **kwargs)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/keystoneauth1/adapter.py", line 248, in request
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     return self.session.request(url, method, **kwargs)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/keystoneauth1/adapter.py", line 248, in request
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     return self.session.request(url, method, **kwargs)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/keystoneauth1/session.py", line 271, in inner
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     return func(*args, **kwargs)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/keystoneauth1/session.py", line 766, in request
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     auth_headers = self.get_auth_headers(auth)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/keystoneauth1/session.py", line 1177, in get_auth_headers
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     return auth.get_headers(self, **kwargs)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/keystoneauth1/plugin.py", line 95, in get_headers
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     token = self.get_token(session)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/keystoneauth1/identity/base.py", line 88, in get_token
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     return self.get_access(session).auth_token
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/keystoneauth1/identity/base.py", line 134, in get_access
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     self.auth_ref = self.get_auth_ref(session)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/keystoneauth1/identity/generic/base.py", line 207, in get_auth_ref
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     self._plugin = self._do_create_plugin(session)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/keystoneauth1/identity/generic/base.py", line 138, in _do_create_plugin
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     authenticated=False)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/keystoneauth1/identity/base.py", line 610, in get_discovery
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     authenticated=authenticated)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/keystoneauth1/discover.py", line 1442, in get_discovery
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     disc = Discover(session, url, authenticated=authenticated)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/keystoneauth1/discover.py", line 526, in __init__
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     authenticated=authenticated)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/keystoneauth1/discover.py", line 101, in get_version_data
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     resp = session.get(url, headers=headers, authenticated=authenticated)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/keystoneauth1/session.py", line 1117, in get
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     return self.request(url, 'GET', **kwargs)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/keystoneauth1/session.py", line 271, in inner
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     return func(*args, **kwargs)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/keystoneauth1/session.py", line 907, in request
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     resp = send(**kwargs)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/keystoneauth1/session.py", line 998, in _send_request
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     resp = self.session.request(method, url, **kwargs)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/requests/sessions.py", line 533, in request
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     resp = self.send(prep, **send_kwargs)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/requests/sessions.py", line 646, in send
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     r = adapter.send(request, **kwargs)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/requests/adapters.py", line 449, in send
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     timeout=timeout
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/urllib3/connectionpool.py", line 667, in urlopen
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/urllib3/connectionpool.py", line 945, in _prepare_proxy
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/urllib3/connection.py", line 345, in connect
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     default_ssl_context = False
2020-12-11 14:56:20.591 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib64/python3.7/ssl.py", line 518, in options
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/urllib3/connection.py", line 345, in connect
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     default_ssl_context = False
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/urllib3/util/ssl_.py", line 277, in create_urllib3_context
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     # (issue #309)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib64/python3.7/ssl.py", line 518, in options
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     super(SSLContext, SSLContext).options.__set__(self, value)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib64/python3.7/ssl.py", line 518, in options
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     super(SSLContext, SSLContext).options.__set__(self, value)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib64/python3.7/ssl.py", line 518, in options
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     super(SSLContext, SSLContext).options.__set__(self, value)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   [Previous line repeated 423 more times]
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors RecursionError: maximum recursion depth exceeded while calling a Python object
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors

[temoto]

Probably duplicate #371 #618

Please post minimal reproduction code.

[thomasgoirand]

Hi Temoto. No, this isn't a duplicate, but a new issue. I'm experiencing this with Python 3.9 under Debian Sid as well, with Eventlet 0.26.1 (with some added patches cherry-picked to fix other issues).

[thomasgoirand]

Hi. I just understood that the issue is triggered with dnspython 2.0.0 (at least for me). Discussing the issue on the relevant already opened bug from now on.