Errors when trying to run high concurrency (10k requests/sec) on linux side, but none on windows side.

[lafftar]

Describe the bug
is_connect and is_request errors when trying to run high concurrency (10k requests submitted at once) on ubuntu, but works fine on windows.

To Reproduce
This is my code:

import asyncio
import math
from random import choice
from time import time

import rnet

from utils.custom_log_format import logger
from utils.increase_limits import set_max_open_files

log = logger(name='RNET')
set_max_open_files()
errors = set()


async def send_request(client, url):
    try:
        resp = await client.get(url)
        return resp.status.as_int()
    except Exception as e:
        if e not in errors:
            log.error(e)
            errors.add(e)
        raise e

async def test_many_clients():
    url = "https://forevercode.online/"
    num_requests = 10_000
    t1 = time()
    clients = [rnet.Client(timeout=60, verify=False) for _ in range(max(1, math.ceil(num_requests / 500)))]  # the verify flag was added to try to get linux to work without errors.
    start_time = asyncio.get_event_loop().time()
    tasks = [asyncio.create_task(send_request(choice(clients), url)) for _ in range(num_requests)]
    results = await asyncio.gather(*tasks, return_exceptions=True)
    end_time = asyncio.get_event_loop().time()
    duration = end_time - start_time
    successful = [r for r in results if not isinstance(r, Exception)]
    num_successful = len(successful)
    req_per_sec = num_successful / duration if duration > 0 else 0

    print(f"rnet: {req_per_sec:.2f} req/sec ({num_successful}/{num_requests} successful)")
    print(f"Statuses: {set(successful)}")
    print(f'Duration: {time() - t1:.2f}s')


if __name__ == "__main__":
    asyncio.run(test_many_clients())

You can just spin up an ubuntu server and run it, or you can pull from my repo: https://github.com/lafftar/requestSpeedTest

These are the types of errors we see:

[/home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/wreq-6.0.0-rc.15/src/error.rs:188:21] err = Error {
    kind: Connect,
    source: Some(
        Failure(
            MidHandshakeSslStream {
                stream: SslStream {
                    stream: PollEvented {
                        io: Some(
                            TcpStream {
                                addr: 207.148.90.202:50096,
                                peer: 149.28.192.18:443,
                                fd: 29,
                            },
                        ),
                    },
                    ssl: Ssl {
                        state: "TLS client read_server_hello",
                        verify_result: Err(
                            X509VerifyError {
                                code: 65,
                                error: "Invalid certificate verification context",
                            },
                        ),
                    },
                },
                error: Error {
                    code: SYSCALL (5),
                    cause: None,
                },
            },
        ),
    ),
    connect_info: None,
}
[2025-09-28 17:29:16,748][RNET]: is_connect error: wreq::Error { kind: Request, uri: https://forevercode.online/, source: Error { kind: Connect, source: Some(Failure(MidHandshakeSslStream { stream: SslStream { stream: PollEvented { io: Some(TcpStream { addr: 207.148.90.202:50096, peer: 149.28.192.18:443, fd: 29 }) }, ssl: Ssl { state: "TLS client read_server_hello", verify_result: Err(X509VerifyError { code: 65, error: "Invalid certificate verification context" }) } }, error: Error { code: SYSCALL (5), cause: None } })), connect_info: None } }
[2025-09-28 17:29:16,803][RNET]: is_request error: wreq::Error { kind: Request, uri: https://forevercode.online/, source: Error { kind: Canceled, source: Some(crate::core::Error(Canceled, "connection closed")), connect_info: Some(Connected { alpn: H2, is_proxied: false, extra: Some(Extra), poisoned: PoisonPill@0x7f36565eaad0 { poisoned: false } }) } }

Expected behavior
A clean run on linux, no errors.

Screenshots

Desktop (please complete the following information):

• Linux OS: Ubuntu 25.04 x64
• Windows OS: Windows 10

Additional context
I own the site and server i'm testing on, i'll leave it on for a day so you can run tests against it if you need, this is the config:

server {
    listen 80;
    server_name forevercode.online www.forevercode.online;
    return 200 $remote_addr;
}

server {
    listen 443 ssl backlog=65535;
    http2 on; # Updated, modern syntax for enabling HTTP/2

    server_name forevercode.online www.forevercode.online;

    # !!! IMPORTANT !!!
    # You must have already run certbot for these files to exist.
    ssl_certificate /etc/letsencrypt/live/forevercode.online/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/forevercode.online/privkey.pem;

    location / {
        default_type text/plain;
        return 200 $remote_addr;
    }
}

I'm doing this to prove to clients that python is fast enough for their needs, I was able to get to 4.7k requests/sec on HTTP only, But I want to get to 15k requests/sec on HTTPS.

[lafftar]

Alright, so a bit of gpt-5 help and figured the verify alone wasn't enough to disable tls, needed to: rnet.Client(timeout=60, verify=False, verify_hostname=False) and now getting this result on linux:

Will work on getting this to work with tls verification on.

[0x676e67]

The underlying socket implementations in Linux and Windows are different, and the specific details can only be understood after careful study.

Windows uses asynchronous I/O represented by IOCP, which performs better than Linux when io_uring is not used. However, the Tokio async runtime used by rnet does not support io_uring.

[lafftar]

A funny case where windows turns out to be more efficient than Linux haha, I guess Tokio doesn't have implementing io_uring as priority.

[0x676e67]

I ran a quick test but couldn’t reproduce what you described.

[lafftar]

Interesting, maybe Debian plays better for some reason, but I was able to hit decent numbers with TLS verification turned off:

I've changed my code setup to try for higher numbers, also trying more cores, we got this result on 2vcpu so far. Will try on 4, then 8 and see what we can squeeze out (afaik rnet scales across cores automatically).

But yeah when I go for 200k request on the 2 core machine we get much of the same errors, I guess you can try pushing the number of requests you're sending to see the same errors...idk much of what's going on in the low level, maybe Debian has a better network stack than Ubuntu or something?

[0x676e67]

Not sure, maybe. Debian is indeed more suitable for servers than Ubuntu. Also, you’ve overestimated the performance of my test machine — it’s just a 2-core 1 GB box, so I can’t handle too many requests. Isn’t this kind of like asking for private paid support?

[lafftar]

Oh sorry, not my intention to infringe on your time, you can ignore me, I'll update the repo with my findings and close it, hopefully in about two days, whether I hit 15k requests/sec or not.

[FayzulSaimun]

Hi, Thank you for this great tool.
I am having same kind of error both in windows and linux. At first I thought this was due to the site anti-bot or something similar. Then I tried with a less secure site and getting the following error:

WARNING HTTP/2 error attempt 1/3: is_request error: wreq::Error { kind: Request, uri: https://www.targetsite.com, source: Error { kind: SendRequest, source: Some(crate::core::Error(Http2, Error { kind: Reset(StreamId(3), INTERNAL_ERROR, Remote) })), connect_info: Some(Connected { alpn: H2, is_proxied: false, extra: Some(Extra), poisoned: PoisonPill@0x5a960120730 { poisoned: false } }) } } | https://www.targetsite.com

My client creation:

        client = Client(
            emulation=EmulationOption.random(),
            tls_info=True,
            allow_redirects=True,
            verify=False, 
            timeout=60,
            connect_timeout=60,
            proxies=(
                [
                    Proxy.all(
                        url=self.proxy_url,
                        username=self.proxy_username,
                        password=self.proxy_password,
                    )
                ]
            ),
        )

Is the issue related to the client or the target host?