Java 21 Virtual Threads: A New Chapter for Asynchronous Programming 🚀

Asynchronous programming becomes the 'backbone' of the system that needs to handle thousands of concurrent requests without wasting resources. It allows the application to continue performing other tasks instead of waiting for I/O operations (reading/writing files, network calls, database queries…).

In Java, the asynchronous model developed through many stages.

=> The limitations of traditional async models: thread wasting, complexity, or debugging challenges, led to the need for a simpler solution. Virtual Threads (from Project Loom) were introduced to offer lightweight, scalable threads without changing coding styles, combining the ease of synchronous code with the efficiency of async performance.

What is a virtual thread?

The virtual thread was introduced in Java 19 (JEP 425) as a preview feature and officially released in Java 21. It was developed by Project Loom with the purpose of helping asynchronous programming to become as effective as synchronous without changing code complexity.

Traditional thread and Virtual Thread

Traditional Thread

Previously we only knew 1 type of thread: traditional thread (platform thread or OS thread). When a task executing request is received, the JVM will create a Java thread and send some information (stack size and thread priority) to the OS to establish 1:1 mapping with the native thread. Once the connection is complete, the Java thread will become an interface to receive operations and delegate them to the underlying native thread for execution. The management and scheduling are handled entirely by the OS. Lead to the JVM job relatively easily.

However, creating native threads in an OS has limitations and is resource-intensive. By default, creating a single thread consumes around ~1MB of memory.

Virtual Thread

Virtual threads have reached a new milestone: the scheduling and execution of tasks are now managed by the JVM instead of the OS as before. Virtual threads no longer maintain a direct 1:1 binding with native threads. Instead, the carrier threads are the ones linked to native threads.

When execution begins, the virtual thread is mounted to the carrier thread. This is made possible by a JVM mechanism called continuation, which will store the start point. The virtual thread then runs normally on the carrier thread, similar to how native threads operate.

In the case of a virtual thread waiting for I/O operations, the carrier thread is unmounted from the virtual thread. The virtual thread’s state is stored in a continuation object on the Java heap, ready to be resumed when needed.

Key Characteristics of Virtual Threads

Virtual Threads is Lightweight

Virtual Threads (also known as Project Loom) in Java are designed to be extremely lightweight—you can create millions of virtual threads without consuming as many resources as traditional threads. This is because a virtual thread does not occupy a dedicated OS thread. Instead, it temporarily "borrows" a carrier thread (a native thread) only when it needs to execute.

Virtual thread is Designed for blocking operation

Tasks like API calls, database queries, file I/O, etc., typically do not continuously consume CPU. Instead, they spend most of their time waiting for external responses. With traditional threads, even while waiting, the thread remains occupied—leading to wasted resources and potential thread exhaustion under high concurrent request loads.

Virtual threads behave differently—when encountering blocking I/O operations (e.g., InputStream.read(), Socket.read()), the virtual thread suspends itself and releases the carrier thread, allowing another virtual thread to use it. This results in:

• Much higher throughput for I/O-bound tasks

• More efficient resource usage (fewer native threads, reduced context switching)

However, the virtual thread doesn’t offer performance gains for CPU-bound task because:

• No Parallelism Gain: Virtual threads still rely on a limited pool of carrier threads (actual OS threads).

• No Extra CPU Cores: If all CPU cores are saturated, adding more virtual threads won’t speed up computation.

• Context Switching Overhead: While virtual threads reduce context-switching costs for I/O, CPU-bound tasks still compete for CPU time.

Virtual threads bring async performance to simple, sync-style code

Traditionally, building scalable applications in Java required using asynchronous programming models like callbacks (easily led to callback-hell), Completable Future, or reactive streams (e.g., Project Reactor, RxJava). These models are harder to read, debug, and maintain, especially for complex business workflows. Virtual threads change that.

They allow developers to write straightforward, sequential code using familiar constructs like try, catch, and for loops — but under the hood, the JVM suspends and resumes virtual threads efficiently whenever they hit a blocking call (e.g., I/O, socket read, database access).

This enables non-blocking behavior with code that looks and feels blocking. In other words: your code can stay "simple and blocking", but your system remains scalable like it's using non-blocking I/O.

Because virtual threads are cheap to create and manage, you can spawn thousands or even millions of them without overwhelming the system — something that would be impractical with platform threads due to OS-level limitations.

Compare between Traditional Thread and Virtual Thread

Virtual threads don't replace the traditional threads that are supplemented to optimize I/O-bound workloads (HTTP requests, DB calls).

Point to Note

• Low cost: Context switch is less expensive; we can create a million virtual threads that spend fewer resources of the OS.

• Backward compatible: The style code is synchronous unchange; only need to replace Thread.start() with Thread.ofVirtual().

• Throughput: Benchmark results show a significant throughput increase compared to traditional threads.

Demo

Goal: Demonstrate virtual threads’ efficiency in high-concurrency blocking operations

Tools & Setup

• Comparison: Virtual Threads vs. Platform Threads (Thread usage, execution time).

• Stack: JDK 21

We will demonstrate the performance differences between traditional (platform) threads and virtual threads using a simple code snippet. The test involves:

• Creating 10,000 threads for both Virtual and Platform, with a simulated I/O wait time of 1s.

• Creating 100,000 threads for both Virtual and Platform, with a simulated I/O wait time of 1s.

• Simulating both CPU-bound operations and I/O-bound operations for a comprehensive comparison

The benchmark results compare virtual threads vs. platform threads across I/O-bound and CPU-bound workloads on a 16-core Windows 11 system. Below is a structured performance analysis:

Performance Matrix Table

Key Observations

1. Virtual Threads Are Dramatically Faster for I/O Tasks

• 10,000 I/O Tasks: Virtual Threads finished in 1,449ms vs. 3,963ms for Platform Threads (2.7x faster).

• 100,000 I/O Tasks: Virtual Threads completed in 4,200ms, while Platform Threads took 35,271ms (8.4x faster).

• Virtual Threads avoid OS thread blocking during I/O operations. Instead of holding an OS thread idle, they yield and resume efficiently, allowing the JVM to handle concurrency with minimal overhead. Platform threads, on the other hand, suffer from thread contention and scheduling delays at high concurrency.

2. Virtual Threads Scale Effortlessly to High Concurrency

• PeakThreadCount for 100k I/O Tasks:

○ Virtual Threads: 31 (almost unchanged from idle state).

○ Platform Threads: 3,786 (near OS limits).

• TotalStartedThreadCount for 100k I/O Tasks:

○ Virtual Threads: 33 (extremely low).

○ Platform Threads: 100,014 (1:1 per task).

○ Virtual threads are lightweight and managed by the JVM rather than the OS. They reuse a small pool of carrier threads, while platform threads consume native OS resources, leading to diminishing returns at scale.

3. Lower System CPU Overhead with Virtual Threads

• 100k I/O Tasks:

○ Virtual Threads: 2.07% System CPU.

○ Platform Threads: 9.52% System CPU.

○ Virtual threads reduce kernel scheduling overhead because they don’t rely on OS thread management. Platform threads force the OS to handle thousands of thread context switches, increasing system load.

4. Memory Trade-off: Virtual Threads Use More but Scale Better

• 100k I/O Tasks:

○ Virtual Threads: 177MB used.

○ Platform Threads: 38MB used.

○ Virtual threads allocate more memory per thread but avoid the exponential slowdown seen with platform threads. The trade-off is justified—8.4x faster execution at 100k tasks is far more valuable than saving memory.

Conclusion

Virtual threads deliver higher performance for I/O-bound tasks, making them ideal for handling HTTP requests and database calls. Conversely, for CPU-bound workloads like heavy calculations, algorithms, or machine learning, traditional threads still outperform them. Additionally, virtual threads are easy to use. They require minimal code changes and follow a synchronous coding style but provide asynchronous-like performance. They’re also compatible with existing synchronous libraries (e.g., JDBC, Servlet). However, the project must upgrade your project to Java 21+ to use virtual threads.

Written by Mr. Phat Le & Mr. Phu Ngo - IT Division at Home Credit Vietnam