Node.js Single-Thread Model Exploration

Node.js adopts the event-driven and asynchronous I/O approach, achieving a single-threaded, highly concurrent JavaScript runtime environment. Since a single thread means only one thing can be done at a time, how does Node.js achieve high concurrency and asynchronous I/O with just one thread? This article will explore the single-threaded model of Node.js around this question.

High Concurrency Strategies

Generally, the solution for high concurrency is to provide a multi-threaded model. The server assigns one thread to each client request and uses synchronous I/O. The system makes up for the time cost of synchronous I/O calls through thread switching. For example, Apache uses this strategy. Given that I/O operations are usually time-consuming, it's difficult to achieve high performance with this approach. However, it is very simple and can implement complex interaction logics.

In fact, most web server sides don't perform much computation. After receiving requests, they pass the requests to other services (such as reading databases), then wait for the results to come back, and finally send the results to the clients. Therefore, Node.js uses a single-threaded model to handle this situation. Instead of assigning a thread to each incoming request, it uses a main thread to handle all requests and then processes I/O operations asynchronously, avoiding the overhead and complexity of creating, destroying threads, and switching between threads.

Event Loop

Node.js maintains an event queue in the main thread. When a request is received, it's added to this queue as an event, and then it continues to receive other requests. When the main thread is idle (no requests are incoming), it starts to loop through the event queue to check if there are events to be processed. There are two cases: for non-I/O tasks, the main thread will handle them directly and return to the upper layer via a callback function; for I/O tasks, it will take a thread from the thread pool to handle the event, specify a callback function, and then continue to loop through other events in the queue.

Once the I/O task in the thread is completed, the specified callback function is executed, and the completed event is placed at the end of the event queue, waiting for the event loop. When the main thread loops to this event again, it directly processes it and returns it to the upper layer. This process is called the Event Loop, and its operating principle is shown in the figure below:

This figure shows the overall operating principle of Node.js. From left to right and from top to bottom, Node.js is divided into four layers: the application layer, the V8 engine layer, the Node API layer, and the LIBUV layer.

• Application Layer: It is the JavaScript interaction layer. Common examples are Node.js modules like http and fs.
• V8 Engine Layer: It uses the V8 engine to parse JavaScript syntax and then interacts with the lower-layer APIs.
• Node API Layer: It provides system calls for the upper-layer modules, usually implemented in C, and interacts with the operating system.
• LIBUV Layer: It is a cross-platform underlying encapsulation that realizes event loops, file operations, etc., and is the core of Node.js for achieving asynchrony.

Whether on the Linux platform or the Windows platform, Node.js internally uses the thread pool to complete asynchronous I/O operations, and LIBUV unifies the calls for different platform differences. So, the single thread in Node.js only means that JavaScript runs in a single thread, not that Node.js as a whole is single-threaded.

Working Principle

The core of Node.js achieving asynchrony lies in events. That is, it treats every task as an event and then simulates the asynchronous effect through the Event Loop. To understand and accept this fact more concretely and clearly, we use pseudocode to describe its working principle below.

1. Define the Event Queue

Since it's a queue, it's a first-in, first-out (FIFO) data structure. We use a JS array to describe it as follows:

/**
 * Define the event queue
 * Enqueue: push()
 * Dequeue: shift()
 * Empty queue: length === 0
 */
let globalEventQueue = [];

We use the array to simulate the queue structure: the first element of the array is the head of the queue, and the last element is the tail. push() inserts an element at the end of the queue, and shift() removes an element from the head of the queue. Thus, a simple event queue is achieved.

2. Define the Request Reception Entrance

Every request will be intercepted and enter the processing function, as shown below:

/**
 * Receive user requests
 * Every request will enter this function
 * Pass parameters request and response
 */
function processHttpRequest(request, response) {
    // Define an event object
    let event = createEvent({
        params: request.params, // Pass request parameters
        result: null, // Store request results
        callback: function() {} // Specify a callback function
    });

    // Add the event to the end of the queue
    globalEventQueue.push(event);
}

This function simply packages the user's request as an event and puts it into the queue, then continues to receive other requests.

3. Define the Event Loop

When the main thread is idle, it starts to loop through the event queue. So we need to define a function to loop through the event queue:

/**
 * The main body of the event loop, executed by the main thread when appropriate
 * Loop through the event queue
 * Handle non-IO tasks
 * Handle IO tasks
 * Execute callbacks and return to the upper layer
 */
function eventLoop() {
    // If the queue is not empty, continue to loop
    while (this.globalEventQueue.length > 0) {
        // Take an event from the head of the queue
        let event = this.globalEventQueue.shift();

        // If it's a time-consuming task
        if (isIOTask(event)) {
            // Take a thread from the thread pool
            let thread = getThreadFromThreadPool();
            // Hand it over to the thread to handle
            thread.handleIOTask(event);
        } else {
            // After handling non-time-consuming tasks, directly return the result
            let result = handleEvent(event);
            // Finally, return to V8 through the callback function, and then V8 returns to the application
            event.callback.call(null, result);
        }
    }
}

The main thread continuously monitors the event queue. For I/O tasks, it hands them over to the thread pool to handle, and for non-I/O tasks, it handles them itself and returns.

4. Handle I/O Tasks

After the thread pool receives the task, it directly processes the I/O operation, such as reading the database:

/**
 * Handle IO tasks
 * Add the event to the end of the queue after completion
 * Release the thread
 */
function handleIOTask(event) {
    // Current thread
    let curThread = this;

    // Operate the database
    let optDatabase = function (params, callback) {
        let result = readDataFromDb(params);
        callback.call(null, result);
    };

    // Execute the IO task
    optDatabase(event.params, function (result) {
        // Store the return result in the event object
        event.result = result;

        // After the IO is completed, it's no longer a time-consuming task
        event.isIOTask = false;

        // Re-add this event to the end of the queue
        this.globalEventQueue.push(event);

        // Release the current thread
        releaseThread(curThread);
    });
}

When the I/O task is completed, the callback is executed, the request result is stored in the event, and the event is put back into the queue, waiting for the loop. Finally, the current thread is released. When the main thread loops to this event again, it directly processes it.

Summarizing the above process, we find that Node.js only uses one main thread to receive requests. After receiving requests, it doesn't process them directly but puts them into the event queue and then continues to receive other requests. When it's idle, it processes these events through the Event Loop, thus achieving the asynchronous effect. Of course, for I/O tasks, it still needs to rely on the thread pool at the system level to handle.

Therefore, we can simply understand that Node.js itself is a multi-threaded platform, but it processes tasks at the JavaScript level in a single thread.

CPU-Intensive Tasks Are a Shortcoming

By now, we should have a simple and clear understanding of the single-threaded model of Node.js. It achieves high concurrency and asynchronous I/O through the event-driven model. However, there are also things that Node.js isn't good at.

As mentioned above, for I/O tasks, Node.js hands them over to the thread pool for asynchronous processing, which is efficient and simple. So, Node.js is suitable for handling I/O-intensive tasks. But not all tasks are I/O-intensive. When encountering CPU-intensive tasks, that is, operations that only rely on CPU calculations, such as data encryption and decryption (node.bcrypt.js), data compression and decompression (node-tar), Node.js will handle them one by one. If the previous task isn't completed, the subsequent tasks can only wait. As shown in the figure below:

In the event queue, if the previous CPU calculation tasks aren't completed, the subsequent tasks will be blocked, resulting in slow response. If the operating system is single-core, it might be tolerable. But now most servers are multi-CPU or multi-core, and Node.js only has one EventLoop, meaning it only occupies one CPU core. When Node.js is occupied by CPU-intensive tasks, causing other tasks to be blocked, there are still CPU cores left idle, resulting in a waste of resources.

So, Node.js is not suitable for CPU-intensive tasks.

Application Scenarios

• RESTful API: Requests and responses only require a small amount of text and don't need much logical processing. Therefore, tens of thousands of connections can be processed concurrently.
• Chat Service: It's lightweight, has high traffic, and doesn't have complex calculation logics.

Leapcell: The Next-Gen Serverless Platform for Web Hosting, Async Tasks, and Redis

Finally, let me introduce the platform that is most suitable for deploying Node.js services: Leapcell.

1. Multi-Language Support

• Develop with JavaScript, Python, Go, or Rust.

2. Deploy unlimited projects for free

• Pay only for usage — no requests, no charges.

3. Unbeatable Cost Efficiency

• Pay-as-you-go with no idle charges.
• Example: $25 supports 6.94M requests at a 60ms average response time.

4. Streamlined Developer Experience

• Intuitive UI for effortless setup.
• Fully automated CI/CD pipelines and GitOps integration.
• Real-time metrics and logging for actionable insights.

5. Effortless Scalability and High Performance

• Auto-scaling to handle high concurrency with ease.
• Zero operational overhead — just focus on building.

Explore more in the documentation!

Leapcell Twitter: https://x.com/LeapcellHQ