Virtual threads did not make anything faster

by Kaushal Bhatt
4 min read

"We moved to virtual threads and the app got faster." I have heard versions of this since Java 21 shipped, and it is not quite what happened. Virtual threads did not make any single request faster. They removed a bottleneck that was making your requests wait. The distinction matters, because it tells you in advance whether your workload will see anything at all. ## Where the wait was coming from A platform thread is expensive — roughly a megabyte of stack, and the OS scheduler has to care about every one of them. So you cap the pool. Say 200 threads. Once 200 requests are blocked on I/O, request 201 waits in a queue while the CPU sits nearly idle. That queueing wait was your "slowness," and the CPU was never the reason for it. A virtual thread costs a few hundred bytes and unmounts from its carrier the moment it blocks on I/O. You can have a hundred thousand of them in flight over a handful of carrier threads. Request 201 no longer queues behind a pool limit; it just runs.

java
// The whole migration, in Spring Boot 3.2+
spring.threads.virtual.enabled=true

## So what actually changes - Per-request latency at low concurrency: unchanged. There was no queue to remove. - Throughput ceiling: dramatically higher, but only when the workload is I/O-bound and was previously thread-starved. Two places this helps you not at all: - CPU-bound work. You still have the same number of cores. Nothing about virtual threads creates compute. - A workload that was never pool-starved. If your 200-thread pool was never full, there was no queueing wait to eliminate, and you will see no difference. ## The caveat that keeps moving synchronized blocks used to pin a virtual thread to its carrier for the duration, cancelling the benefit for that section entirely. That was fixed in JDK 24 by JEP 491 — worth confirming which JDK you are actually running before you assume the old advice applies to you. Half the guidance written between 2023 and 2025 is now describing a JVM you are not on. ## The habit If throughput did not move after you switched, do not go looking for a virtual-threads misconfiguration. Go and check whether your workload was ever thread-pool-limited in the first place. The general version: before adopting a change that removes a constraint, confirm that the constraint was the thing binding you. Otherwise you have done a migration and bought a null result, and you will spend a week looking for the bug in a system that is behaving correctly.