Asynchronous Handlers

[The-EDev]

This is a comment made by u/aninteger on reddit

Since you are using asio I think you really need to be writing async handlers. You can't just set multi-threaded mode on and write handlers that do blocking calls because Crow doesn't appear to be a http server library setup with a large thread pool.

And he/she is correct. Crow does not have non blocking handlers. It would also be a nice feature to have, I'll have to look into it however since I don't even know where to begin working on this.

[The-EDev]

Update: Looking into the issue further, it seems that the individual threads can be blocked indefinitely. This means that Crow could be frozen for more time than timeout would normally allow, while keeping all connections assigned to the blocked thread on hold.

[The-EDev]

Extra context

Currently Crow's threading structure is as follows:
An Acceptor thread and n worker threads are initialized when the server starts. Each having its own io_context (io_service in code).

The Worker threads are constantly running io_context.run() and checking if it returns 0 (which would mean no more work to do).

The Acceptor thread creates a new connection object, and provides it with a selected io_context which it will then use to create a socket object to communicate with a potential client. Once a client is connected to the server, The Acceptor thread will move it over to the created connection and repeat the first step.

The bit most relevant to this issue

The Acceptor thread relies on io_context.post() to make sure the connection code is run through the worker thread. It is my assumption that when io_context.post() is called multiple times, one connection would need to finish executing before the next connection would start.

Considering the fact that connection code execution is not asynchronous (Instead going through a loop of reading, processing, and writing. Which doesn't finish until the object destroys itself when the client disconnects), A worker thread is by all accounts can be and is in some cases blocked by a connection's code.

This becomes a problem when a handler (AKA Route) has to run any code that doesn't return instantly -due to being asynchronous- has to block the entire worker thread (and by extension the other connections).

It is also a problem when considering the fact that any heavy operations will also need to run on (and block) the worker thread when they can be delegated to a separate thread and handled asynchronously.

The proposed solution

My suggestion would be for a worker to manage its connections.

• Each connection should have a state (start, run, suspend, or ready).
• instead of immediately running connection.start(), the connection should be added to a queue managed by the worker thread.
• The worker thread dequeues a connection and starts it
• The connection is either run until finished, or is suspended by the handler
• Once a connection is suspended the thread needs to be re-enqueued and the rest of the queue needs to be handled
• The connection needs to be re-enqueued every time it is dequeued until its status is changed to ready
• Once ready, the connection's resume function can be called to pick up where the handler left off (most likely done through coroutines, which both Boost and ASIO have implementations of)

[philhzss]

Commenting here instead of creating a new issue as I think my question is related to this, but I'm fairly new to all of these terms so sorry if I don't have the proper terminology or understanding

Successfully got Crow to work in my program but I had to created an std::thread (which I had never done, I hadn't tried anything multithreading until this point). It's working great now! But I was wondering, do I have to use multithreading with Crow?? Originally I tried to putting the route/handle in my main loop, but as soon as my program hits the Crow part it stops everything and just waits for a new request. I suppose that's what's referred to as "blocking" vs "non-blocking", not sure though, but is it possible to have Crow be "non-blocking"?

For example would it possible right now to just insert some sort of "check" each time my program loops to see if Crow received a new request since the last loop, and if so return the proper value? Or for now the only way is to have it "wait" for a request to come in (thus doing it in another thread is the best way to have it not block the main thread)?

Just trying to understand more, Thank you!

[luca-schlecker]

No worries @philhzss, I understood your point.

Nice, glad to hear that you got it working, even though you had to deal with something new. Crow currently manages its own threads for accepting and handling messages and will block when app.run() is called, so no, you currently have to create a new thread (using the thread class or std::async in combination with std::launch::async) on your own in order to make it non-blocking.

You got it, a blocking function call will not return until it's done, meanwhile, a non-blocking function call will instantly return (this likely involves blocking function calls inside their own thread).

Maybe that's a feature worth introducing @The-EDev? I mean, it's a pretty common use case. Maybe something like run_async().

[The-EDev]

@luca-schlecker I was just writing my own explanation but you explained all the points I was going to (EVEN THE run_async SUGGESTION!! XD).

The one thing I would add is an example on how to run Crow asynchronously from our own unit tests: auto _ = async(launch::async, [&] { app.port(45451).run(); });

[philhzss]

Thank you very much @luca-schlecker and @The-EDev for the explanations! I have never used async either so that's interesting, I'll read about it. For now thought my 2-threated option is working fine so I'll keep it that way. Thanks for the info though, always great to learn!!