Scale Node JS App using Cluster Module

📍 NodeJS Concurrency

🙂 When It comes to writing Backend (or API for say) we can code them really fast in NodeJS. But do you know that NodeJS is single Threaded 🥲. Yes, It is a Single Threaded Technology that makes our applications not so efficient.

😌 But We can use Asynchronous Programming in NodeJS (Or JS in general) which can make concurrent programming easier. There is something called event loop and Callback Queue.

😊 Like here, setTimeout() is an async function that will not pause the execution of further lines of code (Which is Synchronous) and It will run after 2 seconds as mentioned inside the function. This function will go to the callback queue and will remain there until Call Stack will be empty. Event loop will keep looping and taking the async functions from the Callback queue to the Call stack.

😅 So, This process of sync-async programming gives us the illusion of multithreading or concurrent program but actually it is not.

📍 Using Cluster Module

Now, we know the problem is we can’t use multiple threads handling different processes using NodeJS. But there is a simple hack to this. We can use the Cluster module to run our server or API on multiple CPUs (multi-core CPUs). It will make our server efficient. We’ll also see the matrices of improvement by giving some Load to the server by using Cluster Module and by not using it.

📍 Implementation

Note:- You can find the complete Code in this Git Repo.

• Using Clustering or Multi CPUs

const express = require('express');
const cluster = require('cluster');
const os = require('os');

const app = express();
const numCpus = os.cpus().length;
const PORT = 3000;

app.get('/', (req, res)=>{
    for (let i = 0; i < 1e8; i++) {
        //   
    }
    res.send(`OK :- ${process.pid}`);
    // cluster.worker.kill(); 
})

if(cluster.isMaster){
    for (let i = 0; i < numCpus; i++) {
        cluster.fork();
    }
    cluster.on('exit', (worker, code, signal)=>{
        console.log(`worker ${worker.process.pid} died`);
        cluster.fork();
    })
}else{
    app.listen(PORT, ()=>console.log(`Server Working with pid ${process.pid} at localhost with port `+PORT));
}

// app.listen(PORT, ()=>console.log("Server Working at localhost with port "+PORT));

• Our Simple Program without clustering

const express = require('express');
const cluster = require('cluster');
const os = require('os');

const app = express();
// const numCpus = os.cpus().length;
const PORT = 3000;

app.get('/', (req, res)=>{
    for (let i = 0; i < 1e8; i++) {
        //   
    }
    res.send(`OK :- ${process.pid}`);
    // cluster.worker.kill();
})

app.listen(PORT, ()=>console.log("Server Working at localhost with port "+PORT));

👉 The important thing in the clustering code is to understand that we are checking if the program is master one? If so then we are making our servers equal to our CPU cores that will run on different cores. Note Here, fork() does make a child process.

I have 8 Core CPU and you can see 8-instances of my server up and running

Note:- To test the load we are using loadtest here. Command:- loadtest -n 1000 -c 100 http://localhost:3000/

📍 Load Testing

• Using Cluster Module

• Using Simple Program

🎉 You can clearly see the difference in latency whether it is the mean latency or Percentage of the requests served within a certain time

😇 Note:- Now, You can make your program efficient using Cluster Module. But don’t forget to load test the program.

Thanks for Reading, Hope it helped in some way. Follow for more in depth and on point tutorials.