Bot fails when deploying with webhooks on gcloud

[jiwidi]

Steps to reproduce

Steps linked in this tutorial: https://seminar.io/2018/09/03/building-serverless-telegram-bot/

Expected behaviour

The bot should be correctly deployed on gcloud functions and act as as mirror bot.

Actual behaviour

The bot fails on a simple commnad:

def webhook(request):
    bot = telegram.Bot(token=os.environ["TELEGRAM_TOKEN"])
    if request.method == "POST":
        update = telegram.Update.de_json(request.get_json(force=True), bot)
        chat_id = update.message.chat.id
        # Reply with the same message
        bot.sendMessage(chat_id=chat_id, text=update.message.text)
    return "ok"

It seems to me that it fails instanciation the connection. Could be a urrlib issue as well.

I tried using urllib3==1.20 as I saw it as a solution for other issues but the error persists.

Configuration

Operating System:

Version of Python, python-telegram-bot & dependencies:
python 3.7
python-telegram-bot=1.11.0
urllib3==1.24.2

Logs

Retrying (Retry(total=0, connect=None, read=None, redirect=None)) after connection broken by 'NewConnectionError('<telegram.vendor.ptb_urllib3.urllib3.connection.VerifiedHTTPSConnection object at 0x2b45c2e30b00>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /bot<BOTTOKEN>/sendMessage"

gaierror: [Errno -3] Temporary failure in name resolution During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/env/local/lib/python3.7/site-packages/telegram/vendor/ptb_urllib3/urllib3/connectionpool.py", line 617, in urlopen chunked=chunked) File "/env/local/lib/python3.7/site-packages/telegram/vendor/ptb_urllib3/urllib3/connectionpool.py", line 360, in _make_request self._validate_conn(conn) File "/env/local/lib/python3.7/site-packages/telegram/vendor/ptb_urllib3/urllib3/connectionpool.py", line 857, in _validate_conn super(HTTPSConnectionPool, self)._validate_conn(conn) File "/env/local/lib/python3.7/site-packages/telegram/vendor/ptb_urllib3/urllib3/connectionpool.py", line 289, in _validate_conn conn.connect() File "/env/local/lib/python3.7/site-packages/telegram/vendor/ptb_urllib3/urllib3/connection.py", line 284, in connect conn = self._new_conn() File "/env/local/lib/python3.7/site-packages/telegram/vendor/ptb_urllib3/urllib3/connection.py", line 150, in _new_conn self, "Failed to establish a new connection: %s" % e) telegram.vendor.ptb_urllib3.urllib3.exceptions.NewConnectionError: <telegram.vendor.ptb_urllib3.urllib3.connection.VerifiedHTTPSConnection object at 0x2a652ab239b0>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/env/local/lib/python3.7/site-packages/telegram/utils/request.py", line 203, in _request_wrapper resp = self._con_pool.request(*args, **kwargs) File "/env/local/lib/python3.7/site-packages/telegram/vendor/ptb_urllib3/urllib3/request.py", line 70, in request **urlopen_kw) File "/env/local/lib/python3.7/site-packages/telegram/vendor/ptb_urllib3/urllib3/request.py", line 148, in request_encode_body return self.urlopen(method, url, **extra_kw) File "/env/local/lib/python3.7/site-packages/telegram/vendor/ptb_urllib3/urllib3/poolmanager.py", line 244, in urlopen response = conn.urlopen(method, u.request_uri, **kw) File "/env/local/lib/python3.7/site-packages/telegram/vendor/ptb_urllib3/urllib3/connectionpool.py", line 695, in urlopen **response_kw) File "/env/local/lib/python3.7/site-packages/telegram/vendor/ptb_urllib3/urllib3/connectionpool.py", line 695, in urlopen **response_kw) File "/env/local/lib/python3.7/site-packages/telegram/vendor/ptb_urllib3/urllib3/connectionpool.py", line 695, in urlopen **response_kw) File "/env/local/lib/python3.7/site-packages/telegram/vendor/ptb_urllib3/urllib3/connectionpool.py", line 666, in urlopen _stacktrace=sys.exc_info()[2]) File "/env/local/lib/python3.7/site-packages/telegram/vendor/ptb_urllib3/urllib3/util/retry.py", line 376, in increment raise MaxRetryError(_pool, url, error or ResponseError(cause)) telegram.vendor.ptb_urllib3.urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='api.telegram.org', port=443): Max retries exceeded with url: /bot425150643:AAHNaoHZhkEZCkgVGMg4t4uEWeUAWEoh6RA/sendMessage (Caused by NewConnectionError('<telegram.vendor.ptb_urllib3.urllib3.connection.VerifiedHTTPSConnection object at 0x2a652ab239b0>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/env/local/lib/python3.7/site-packages/google/cloud/functions/worker.py", line 346, in run_http_function result = _function_handler.invoke_user_function(flask.request) File "/env/local/lib/python3.7/site-packages/google/cloud/functions/worker.py", line 217, in invoke_user_function return call_user_function(request_or_event) File "/env/local/lib/python3.7/site-packages/google/cloud/functions/worker.py", line 210, in call_user_function return self._user_function(request_or_event) File "/user_code/main.py", line 11, in webhook bot.sendMessage(chat_id=chat_id, text=update.message.text) File "/env/local/lib/python3.7/site-packages/telegram/bot.py", line 65, in decorator result = func(self, *args, **kwargs) File "/env/local/lib/python3.7/site-packages/telegram/bot.py", line 90, in decorator result = self._request.post(url, data, timeout=kwargs.get('timeout')) File "/env/local/lib/python3.7/site-packages/telegram/utils/req

[tsnoam]

@jiwidi Thanks for the report.

1. Google's App Engine is somewhat of a problem as they have modified socket library, making it troublesome to support and thus we do not provide support for this environment. Nevertheless we try to assist if we can and fix issues resulting from their special system. Community contribution is always welcome as well.
2. In order to use a different urllib3, you will have to replace the directory telegram/vendor/ptb_urllib3. Have you patched our library that way? (Installing "normal" urllib3 will not do the trick) Can you please confirm that replacing that directory mitigates your problem?
3. It is difficult to read the stack trace. Can you please provide it with properly broken lines? That will be of great assistance.

[jiwidi]

Hi @tsnoam, sorry for the late reply.

2. I haven't done that, I actually thought I will just need the "normal" urllib3 so I included it on my requirements.txt (google only install packages listed here". Does your patched library come with telegrambot when installing it with pip? If so, I probably don't need to specify any urlib in the requirements.txt.

3. I agree is difficult but this is the format they provide. Here is my try to format it nicely:

gaierror: [Errno -3] Temporary failure in name resolution During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/env/local/lib/python3.7/site-packages/telegram/vendor/ptb_urllib3/urllib3/connectionpool.py", 
line 617, in urlopen chunked=chunked) File "/env/local/lib/python3.7/site-packages/telegram/vendor/ptb_urllib3/urllib3/connectionpool.py", 
line 360, in _make_request self._validate_conn(conn) File "/env/local/lib/python3.7/site-packages/telegram/vendor/ptb_urllib3/urllib3/connectionpool.py", 
line 857, in _validate_conn super(HTTPSConnectionPool, self)._validate_conn(conn) File "/env/local/lib/python3.7/site-packages/telegram/vendor/ptb_urllib3/urllib3/connectionpool.py", line 289, in _validate_conn conn.connect() File "/env/local/lib/python3.7/site-packages/telegram/vendor/ptb_urllib3/urllib3/connection.py", 
line 284, in connect conn = self._new_conn() File "/env/local/lib/python3.7/site-packages/telegram/vendor/ptb_urllib3/urllib3/connection.py", 
line 150, in _new_conn self, "Failed to establish a new connection: %s" % e) telegram.vendor.ptb_urllib3.urllib3.exceptions.NewConnectionError: <telegram.vendor.ptb_urllib3.urllib3.connection.VerifiedHTTPSConnection object at 0x2a652ab239b0>:  Failed to establish a new connection: [Errno -3] Temporary failure in name resolution During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/env/local/lib/python3.7/site-packages/telegram/utils/request.py", 
line 203, in _request_wrapper resp = self._con_pool.request(*args, **kwargs) File "/env/local/lib/python3.7/site-packages/telegram/vendor/ptb_urllib3/urllib3/request.py", 
line 70, in request **urlopen_kw) File "/env/local/lib/python3.7/site-packages/telegram/vendor/ptb_urllib3/urllib3/request.py", 
line 148, in request_encode_body return self.urlopen(method, url, **extra_kw) File "/env/local/lib/python3.7/site-packages/telegram/vendor/ptb_urllib3/urllib3/poolmanager.py", 
line 244, in urlopen response = conn.urlopen(method, u.request_uri, **kw) File "/env/local/lib/python3.7/site-packages/telegram/vendor/ptb_urllib3/urllib3/connectionpool.py", line 695, in urlopen **response_kw) File "/env/local/lib/python3.7/site-packages/telegram/vendor/ptb_urllib3/urllib3/connectionpool.py", 
line 695, in urlopen **response_kw) File "/env/local/lib/python3.7/site-packages/telegram/vendor/ptb_urllib3/urllib3/connectionpool.py", 
line 695, in urlopen **response_kw) File "/env/local/lib/python3.7/site-packages/telegram/vendor/ptb_urllib3/urllib3/connectionpool.py", 
line 666, in urlopen _stacktrace=sys.exc_info()[2]) File "/env/local/lib/python3.7/site-packages/telegram/vendor/ptb_urllib3/urllib3/util/retry.py", 
line 376, in increment raise MaxRetryError(_pool, url, error or ResponseError(cause)) telegram.vendor.ptb_urllib3.urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='api.telegram.org', port=443): Max retries exceeded with url: /bot425150643:AAHNaoHZhkEZCkgVGMg4t4uEWeUAWEoh6RA/sendMessage (Caused by NewConnectionError('<telegram.vendor.ptb_urllib3.urllib3.connection.VerifiedHTTPSConnection object at 0x2a652ab239b0>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/env/local/lib/python3.7/site-packages/google/cloud/functions/worker.py", line 346, in run_http_function result = _function_handler.invoke_user_function(flask.request) File "/env/local/lib/python3.7/site-packages/google/cloud/functions/worker.py", 
line 217, in invoke_user_function return call_user_function(request_or_event) File "/env/local/lib/python3.7/site-packages/google/cloud/functions/worker.py", 
line 210, in call_user_function return self._user_function(request_or_event) File "/user_code/main.py", line 11, in webhook bot.sendMessage(chat_id=chat_id, text=update.message.text) File "/env/local/lib/python3.7/site-packages/telegram/bot.py", 
line 65, in decorator result = func(self, *args, **kwargs) File "/env/local/lib/python3.7/site-packages/telegram/bot.py", 
line 90, in decorator result = self._request.post(url, data, timeout=kwargs.get('timeout')) File "/env/local/lib/python3.7/site-packages/telegram/utils/req

I'll try to deploy it again with your feedback regarding your patched urlib and get back to you asap.

Thanks

[tsnoam]

I checked gae documentation again which refers to the urlllib3 documentation:
https://urllib3.readthedocs.io/en/latest/advanced-usage.html#google-app-engine
You should try using the socket API by modifying app.yaml:

env_variables:
    GAE_USE_SOCKETS_HTTPLIB : 'true'

[tsnoam]

Also, to make it clear:
You do not need to specify urlllib3 as a requirement. We have a modified version vendored inside python-telegram-bot package.

[jiwidi]

Hi @tsnoam

Adding both GAE_USE_SOCKETS_HTTPLIB : 'true' and removing urllib3 as a requirement has fixed the issue.

If anyone has the same issue you can check the files for my bot here . It's deployed via google cloud functions.

[tsnoam]

Thanks @jiwidi
I've updated the wiki with what we've learned here.
https://github.com/python-telegram-bot/python-telegram-bot/wiki/Where-to-host-Telegram-Bots