NPM No Longer Issues SSL Certificates with Cloudflare when using ISP shared IPv6 address

[F1zzyD]

Checklist

• Have you pulled and found the error with jc21/nginx-proxy-manager:latest docker image?
  • Yes
• Are you sure you're not using someone else's docker image?
  • Yes
• Have you searched for similar issues (both open and closed)?
  • Yes

Describe the bug
NPM can no longer issue SSL certificates with Cloudfare. I receive "Internal Error" or, when using a DNS challenge, the webpage never loads.

Nginx Proxy Manager Version
latest and dev

To Reproduce
Steps to reproduce the behavior:

1. Go to request an SSL Certificate
2. Wait
3. See error "Internal Error"

Expected behavior
An SSL is issued....

Log.
Deleting file: /data/nginx/proxy_host/2.conf
Deleting file: /data/nginx/proxy_host/2.conf.err
Could not delete file: {
"errno": -2,
"code": "ENOENT",
"syscall": "unlink",
"path": "/data/nginx/proxy_host/2.conf.err"
}
CMD: /usr/sbin/nginx -t -g "error_log off;"
Reloading Nginx
CMD: /usr/sbin/nginx -s reload
Requesting Let'sEncrypt certificates for Cert #6: hass.domain.cc
Command: certbot certonly --config "/etc/letsencrypt.ini" --work-dir "/tmp/letsencrypt-lib" --logs-dir "/tmp/letsencrypt-log" --cert-name "npm-6" --agree-tos --authenticator webroot --email "admin@example.com" --preferred-challenges "dns,http" --domains "hass.domain.cc"
Command: certbot certonly --config "/etc/letsencrypt.ini" --work-dir "/tmp/letsencrypt-lib" --logs-dir "/tmp/letsencrypt-log" --cert-name "npm-6" --agree-tos --authenticator webroot --email "admin@example.com" --preferred-challenges "dns,http" --domains "hass.domain.cc"
CMD: certbot certonly --config "/etc/letsencrypt.ini" --work-dir "/tmp/letsencrypt-lib" --logs-dir "/tmp/letsencrypt-log" --cert-name "npm-6" --agree-tos --authenticator webroot --email "admin@example.com" --preferred-challenges "dns,http" --domains "hass.domain.cc"
Deleting file: /data/nginx/temp/letsencrypt_6.conf
CMD: /usr/sbin/nginx -t -g "error_log off;"
Reloading Nginx
CMD: /usr/sbin/nginx -s reload
Saving debug log to /tmp/letsencrypt-log/letsencrypt.log
Some challenges have failed.
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /tmp/letsencrypt-log/letsencrypt.log or re-run Certbot with -v for more details.

Operating System
Rasbian OS headless

Additional context
This worked just fine two days ago but now it's completely broken. I went and asked the community at letsencrypt.org, however they were persistent that lets encrypt was working fine and that the problem was my domain (hass.domain.cc) not being reachable by HTTP, when I want my domain to be reached only by HTTPS.

I followed the steps at #3824 and those steps did not help. The goal is to get a local server hosted on a port sent through NGM and to my subdomain, and finally issued through HTTPS from Cloudflare. I now have 20 subdomains offline because of this.

I'm having the same problem with the latest pull of NPM trying to use Cloudflare



I have extracted the logs from certbot as follows
`[root@docker-3f6b379cb76a:/tmp/certbot-log-ucinqv3a]# cat log
2024-06-27 18:27:49,264:DEBUG:certbot._internal.log:Exiting abnormally:
Traceback (most recent call last):
File "/opt/certbot/lib/python3.11/site-packages/certbot/_internal/plugins/disco.py", line 191, in find_all
cls._load_entry_point(entry_point, plugins)
File "/opt/certbot/lib/python3.11/site-packages/certbot/_internal/plugins/disco.py", line 203, in _load_entry_point
plugin_ep = PluginEntryPoint(entry_point)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/certbot/lib/python3.11/site-packages/certbot/_internal/plugins/disco.py", line 42, in init
self.plugin_cls: Type[interfaces.Plugin] = entry_point.load()
^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/importlib/metadata/init.py", line 202, in load
module = import_module(match.group('module'))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/importlib/init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "", line 1206, in _gcd_import
File "", line 1178, in _find_and_load
File "", line 1149, in _find_and_load_unlocked
File "", line 690, in _load_unlocked
File "", line 940, in exec_module
File "", line 241, in _call_with_frames_removed
File "/opt/certbot/lib/python3.11/site-packages/certbot_dns_cloudflare/_internal/dns_cloudflare.py", line 9, in
import CloudFlare
ModuleNotFoundError: No module named 'CloudFlare'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/opt/certbot/bin/certbot", line 8, in
sys.exit(main())
^^^^^^
File "/opt/certbot/lib/python3.11/site-packages/certbot/main.py", line 19, in main
return internal_main.main(cli_args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/certbot/lib/python3.11/site-packages/certbot/_internal/main.py", line 1866, in main
plugins = plugins_disco.PluginsRegistry.find_all()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/certbot/lib/python3.11/site-packages/certbot/_internal/plugins/disco.py", line 193, in find_all
raise errors.PluginError(
certbot.errors.PluginError: The 'certbot_dns_cloudflare._internal.dns_cloudflare' plugin errored while loading: No module named 'CloudFlare'. You may need to remove or update this plugin. The Certbot log will contain the full error details and this should be reported to the plugin developer.
2024-06-27 18:27:49,264:ERROR:certbot._internal.log:The 'certbot_dns_cloudflare._internal.dns_cloudflare' plugin errored while loading: No module named 'CloudFlare'. You may need to remove or update this plugin. The Certbot log will contain the full error details and this should be reported to the plugin developer.`

[MAMedici]

I'm also having this problem. I was able to get a let's encrypt cert issued two days ago, but now I'm getting various certbot errors. I've been beating my head against this for more than a day now.

CommandError: The 'certbot_dns_cloudflare._internal.dns_cloudflare' plugin errored while loading: No module named 'CloudFlare'. You may need to remove or update this plugin. The Certbot log will contain the full error details and this should be reported to the plugin developer.
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /tmp/certbot-log-j5f8b3u3/log or re-run Certbot with -v for more details.

    at /app/lib/utils.js:16:13
    at ChildProcess.exithandler (node:child_process:430:5)
    at ChildProcess.emit (node:events:519:28)
    at maybeClose (node:internal/child_process:1105:16)
    at ChildProcess._handle.onexit (node:internal/child_process:305:5)

CommandError: The 'certbot_dns_cloudflare._internal.dns_cloudflare' plugin errored while loading: No module named 'CloudFlare'. You may need to remove or update this plugin. The Certbot log will contain the full error details and this should be reported to the plugin developer.
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /tmp/certbot-log-_0j1s87e/log or re-run Certbot with -v for more details.
The 'certbot_dns_cloudflare._internal.dns_cloudflare' plugin errored while loading: No module named 'CloudFlare'. You may need to remove or update this plugin. The Certbot log will contain the full error details and this should be reported to the plugin developer.
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /tmp/certbot-log-91eshejb/log or re-run Certbot with -v for more details.
ERROR: Could not find a version that satisfies the requirement acme== (from versions: 0.0.0.dev20151006, 0.0.0.dev20151008, 0.0.0.dev20151017, 0.0.0.dev20151020, 0.0.0.dev20151021, 0.0.0.dev20151024, 0.0.0.dev20151030, 0.0.0.dev20151104, 0.0.0.dev20151107, 0.0.0.dev20151108, 0.0.0.dev20151114, 0.0.0.dev20151123, 0.0.0.dev20151201, 0.1.0, 0.1.1, 0.2.0, 0.3.0, 0.4.0, 0.4.1, 0.4.2, 0.5.0, 0.6.0, 0.7.0, 0.8.0, 0.8.1, 0.9.0, 0.9.1, 0.9.2, 0.9.3, 0.10.0, 0.10.1, 0.10.2, 0.11.0, 0.11.1, 0.12.0, 0.13.0, 0.14.0, 0.14.1, 0.14.2, 0.15.0, 0.16.0, 0.17.0, 0.18.0, 0.18.1, 0.18.2, 0.19.0, 0.20.0, 0.21.0, 0.21.1, 0.22.0, 0.22.1, 0.22.2, 0.23.0, 0.24.0, 0.25.0, 0.25.1, 0.26.0, 0.26.1, 0.27.0, 0.27.1, 0.28.0, 0.29.0, 0.29.1, 0.30.0, 0.30.1, 0.30.2, 0.31.0, 0.32.0, 0.33.0, 0.33.1, 0.34.0, 0.34.1, 0.34.2, 0.35.0, 0.35.1, 0.36.0, 0.37.0, 0.37.1, 0.37.2, 0.38.0, 0.39.0, 0.40.0, 0.40.1, 1.0.0, 1.1.0, 1.2.0, 1.3.0, 1.4.0, 1.5.0, 1.6.0, 1.7.0, 1.8.0, 1.9.0, 1.10.0, 1.10.1, 1.11.0, 1.12.0, 1.13.0, 1.14.0, 1.15.0, 1.16.0, 1.17.0, 1.18.0, 1.19.0, 1.20.0, 1.21.0, 1.22.0, 1.23.0, 1.24.0, 1.25.0, 1.26.0, 1.27.0, 1.28.0, 1.29.0, 1.30.0, 1.31.0, 1.32.0, 2.0.0, 2.1.0, 2.2.0, 2.3.0, 2.4.0, 2.5.0, 2.6.0, 2.7.0, 2.7.1, 2.7.2, 2.7.3, 2.7.4, 2.8.0, 2.9.0, 2.10.0, 2.11.0)
ERROR: No matching distribution found for acme==

[notice] A new release of pip is available: 24.0 -> 24.1.1
[notice] To update, run: pip install --upgrade pip

    at /app/lib/utils.js:16:13
    at ChildProcess.exithandler (node:child_process:430:5)
    at ChildProcess.emit (node:events:519:28)
    at maybeClose (node:internal/child_process:1105:16)
    at ChildProcess._handle.onexit (node:internal/child_process:305:5)

[sithypenguin]

Jumping on this train myself. I have been running NPM on portainer for the last year or so and ran into the issue a day or two ago. I thought it was something on my side so I spun up a new VM and installed docker and portainer on it. Same error message as before

6/28/2024] [12:28:48 AM] [Global   ] › ⬤  debug     CMD: rm -f '/etc/letsencrypt/credentials/credentials-1' || true
[6/28/2024] [12:28:48 AM] [Global   ] › ⬤  debug     CMD: /usr/sbin/nginx -t -g "error_log off;"
[6/28/2024] [12:28:49 AM] [Nginx    ] › ℹ  info      Reloading Nginx
[6/28/2024] [12:28:49 AM] [Global   ] › ⬤  debug     CMD: /usr/sbin/nginx -s reload
[6/28/2024] [12:28:49 AM] [Express  ] › ⚠  warning   The 'certbot_dns_cloudflare._internal.dns_cloudflare' plugin errored while loading: No module named 'CloudFlare'. You may need to remove or update this plugin. The Certbot log will contain the full error details and this should be reported to the plugin developer.

[328902301]

I encountered the same issue as well. Two to three days ago, I purchased a new server and successfully applied for a certificate for a .top domain using Cloudflare's API on the server. Yesterday, I rebuilt the server system and reinstalled the certificate, but the application failed. To verify the source of the problem, I tried to use the certificate on another server that is still in use, but it also failed. After a day of investigation, I determined that the issue is related to the .top domain. Applying for a certificate for such a domain results in the aforementioned error code, while .xyz certificates can be successfully applied in the same environment.

[hehe9011]

I encountered the same issue as well. Two to three days ago, I purchased a new server and successfully applied for a certificate for a .top domain using Cloudflare's API on the server. Yesterday, I rebuilt the server system and reinstalled the certificate, but the application failed. To verify the source of the problem, I tried to use the certificate on another server that is still in use, but it also failed. After a day of investigation, I determined that the issue is related to the .top domain. Applying for a certificate for such a domain results in the aforementioned error code, while .xyz certificates can be successfully applied in the same environment.

confirmed, my .top domain is unworkable, but my .cn works, just the same environment.

[SkyEnders942]

It was working fine not long ago, but now it's not working. I've been trying to fix it for days, but I haven't been successful.😭😭

[stathismes]

Someone solved it on Reddit (FYI @jc21):

https://www.reddit.com/r/nginxproxymanager/comments/1dpox5n/comment/laky9kk/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button

[328902301]

There are currently two bugs, one is the NPM version issue, the latest version may present the problem mentioned in the code, it's fine to downgrade to a lower version or use the latest test version, just a heads up, there might be network issues in Mainland China, which was just discovered during the testing process and hasn't been encountered before. The second is the issue with the .top domain, it seems no one has solved it yet, only waiting for the official fix.

[sithypenguin]

Someone solved it on Reddit (FYI @jc21):

https://www.reddit.com/r/nginxproxymanager/comments/1dpox5n/comment/laky9kk/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button

Just did the steps outlined and they worked for me!

[SkyEnders942]

Someone solved it on Reddit (FYI @jc21):

https://www.reddit.com/r/nginxproxymanager/comments/1dpox5n/comment/laky9kk/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button

Thank you, it worked for me.

[Hackpig1974]

I was able to get this working as well. Thanks for the heads up.

** Update *** - Adding the steps from Reddit in case something happens:

Got it to work! Using 2.11.1 (may work on latest, but I rolled back to 2.11.1 and it works).

Follow these steps:

docker exec -it /bin/bash
Run certbot and see it errors out saying that cloudflare-dns plugin is causing an issue (this is to confirm you have the issue I was seeing that was fixed by this method)
pip uninstall certbot-dns-cloudflare
Run certbot and see it does not error out
pip install certbot-dns-cloudflare
Run certbot and see it does not error out
Try creating a SSL cert and it should work

For future people, if for some reason this doesn't work try doing

pip uninstall certbot-dns-cloudflare
pip install --upgrade pip
and the reinstall the plugin but specify the version
pip install --force-reinstall "certbot-dns-cloudflare==2.11.0"
and restart the container.

For some reason even if you reinstall the plugin after upgrading pip it will always pull the version that it was installed.

[F1zzyD]

I was able to get this working as well. Thanks for the heads up.

** Update *** - Adding the steps from Reddit in case something happens:

Got it to work! Using 2.11.1 (may work on latest, but I rolled back to 2.11.1 and it works).

Follow these steps:

docker exec -it /bin/bash Run certbot and see it errors out saying that cloudflare-dns plugin is causing an issue (this is to confirm you have the issue I was seeing that was fixed by this method) pip uninstall certbot-dns-cloudflare Run certbot and see it does not error out pip install certbot-dns-cloudflare Run certbot and see it does not error out Try creating a SSL cert and it should work

For future people, if for some reason this doesn't work try doing

pip uninstall certbot-dns-cloudflare pip install --upgrade pip and the reinstall the plugin but specify the version pip install --force-reinstall "certbot-dns-cloudflare==2.11.0" and restart the container.

For some reason even if you reinstall the plugin after upgrading pip it will always pull the version that it was installed.

This does not work on v2.11.1 or the latest image. I have tested this on two systems that were wiped clean. Still results in "Internal Error".

[Hackpig1974]

Not sure what is going on for you, but I was able to get it working with these steps specifically.

docker exec -it nginx-app-1 /bin/bash
pip uninstall certbot-dns-cloudflare
pip install --upgrade pip
and the reinstall the plugin but specify the version
pip install --force-reinstall "certbot-dns-cloudflare==2.11.0"
and restart the container.

My NPM version in the bottom left says: v2.11.2 © 2024Theme by [Tabler]

---

On Login to the container I am prompted with:
Version 2.11.2 (12d77e3) 2024-05-22 22:49:17 UTC, OpenResty 1.21.4.3, debian 12 (bookworm), Certbot certbot 2.11.0

and: pip list | grep cloud
certbot-dns-cloudflare 2.11.0
cloudflare 2.19.4

I'm curious if your message is exactly the same, or different. I can check some versions if that helps, just let me know.

[MAMedici]

Removing and adding the certbot-dns-cloudflare fixed the problem for me.