Explain that enabling virtual threads disables traditional thread pools

[stsypanov]

While investigating this issue https://stackoverflow.com/questions/78790376/spring-boot-application-gets-stuck-when-virtual-threads-are-used-on-java-21 I've found out, that the property

server:
  tomcat:
    threads:
      max: 110

works only for platform threads and doesn't affect amount of carrier threads in ForkJoinPool when

spring:
  threads:
    virtual:
      enabled: true

the only way to change their amount is to use -Djdk.virtualThreadScheduler.parallelism=n which seems to be wrong in this case as it would affect all the JVM, not one particular ForkJoinPool. See https://stackoverflow.com/questions/78815247/can-i-configure-parallelism-of-forkjoinpool-created-for-tomcat-in-spring-boot-ap/78815410

[bclozel]

I'm not sure I understand what's being suggested here.

Is this about setting jdk.virtualThreadScheduler.parallelism to server.tomcat.threads.max's value? As you have pointed out, this pool of platform threads is not just used by Tomcat and is meant to be used by the entire application for all virtual threads for efficiency. Doing so would impact not just Tomcat but the entire application and unrelated virtual threads usage.

Maybe you're suggesting to somehow configure a different ForkJoinPool for Tomcat with server.tomcat.threads.max as its platform thread count. I think this would make things worse when it comes to performance and memory usage (and I don't even know if this is possible right now).

In all cases, using server.tomcat.threads.max's value to size a any platform thread pool for virtual threads is bound to be counter-productive as the default value for this property is 200. This means that anyone enabling virtual threads with Tomcat would have 200 platform threads, turning their virtual threads experience into something unusable.

Right now the property is documented like so: server.tomcat.threads.max: Maximum amount of worker threads.. Maybe we can tweak its description to make this clearer? All other solutions seem quite counter-productive to me.

Looking at the StackOverflow question, it seems that the root problem is about thread pinning, which can happen in any library using virtual threads (not just Tomcat) and impact the entire application. I believe this issue should not be fixed by sizing the platform thread pool but rather using the tools provided by the JVM to find the problem.

[stsypanov]

Maybe you're suggesting to somehow configure a different ForkJoinPool for Tomcat with server.tomcat.threads.max as its platform thread count

This seems to be the most logical to me, as it would result in the same amount of platform threads regardless of whether Spring Boot is run with or without virtual threads.

Another option is to split server.tomcat.threads.max in two:

1. for the case virtual threads are off and threadx.platform.max denotes amount of platform threads
2. for the case virtual threads are on and threadx,carrier.max denotes amount of carrier threads, configuring one particular ForkJoinPool used by Tomcat exclusively

Maybe we can tweak its description to make this clearer

You mean explicitly specifying that the option denotes amount of platform threads?

[bclozel]

This seems to be the most logical to me, as it would result in the same amount of platform threads regardless of whether Spring Boot is run with or without virtual threads.

With spring.threads.virtual.enabled=true Spring Boot, not only sets up Tomcat for Virtual Threads, but all libraries where we can apply this. The goal behind virtual threads is to use a single pool of platform threads for the entire application (usually sized after the CPU count) and stop wasting server resources when waiting for blocking I/O. Doing so would undo one of the main purposes of virtual threads: decreasing memory usage. Thread pools are merely a way to work around the problem solved by virtual threads. So while the thread count would be consistent, the entire feature would be made useless.

As for splitting server.tomcat.threads.max, I don't think Tomcat properties should be involved in setting something that is meant to be JVM-wide.

You mean explicitly specifying that the option denotes amount of platform threads?

No, I meant making the description a bit more explicit by saying that those worker threads belong to a thread pool, which is irrelevant once virtual threads are enabled. I'm not sure how to make this more obvious from the property description. Maybe we can expand this part of our reference documentation to say that where virtual threads are supported, traditional thread pools (and their properties) will be skipped entirely.

[stsypanov]

I meant making the description a bit more explicit by saying that those worker threads belong to a thread pool, which is irrelevant once virtual threads are enabled

Sounds reasonable.

Maybe we can expand this part of our reference documentation to say that where virtual threads are supported, traditional thread pools (and their properties) will be skipped entirely

Description of server.tomcat.threads.max should be extended either, I guess. For now it is Maximum amount of worker threads., which is applicable only for disabled virtual threads.