Skip to main content

Enhancing Node.js Microservices Performance through Connection Keep-Alive

Greetings!

Efficient performance is a crucial non-functional requirement in modern applications. Customers can become easily frustrated and may seek alternative solutions if performance falls short. As developers, it is our responsibility to ensure that performance meets the required standards. I'd like to highlight a common mistake that developers might make if they are not familiar with the underlying theory. While the focus here is primarily on Node.js, the principles apply more broadly.

Microservices leverage diverse technologies, and Node.js is a good candidate for API Gateways/ Proxy servers. These gateways play a key role in routing traffic to the underlying downstream microservices. Understanding this dynamic is both interesting and essential when working with Node.js.

Note: In Node.js 19, agent keep-alive is now enabled by default.

The Problem

When incorporating libraries, developers often stick to default configurations or modify only the apparent settings. This approach is suboptimal when dealing with HTTP connections in Node.js because the Node.js HTTP agent does not automatically maintain HTTP connections. This oversight is crucial, as utilizing Node.js for downstream HTTP calls without connection persistence can lead to sluggish performance and excessive resource utilization.
  1. DNS resolution
  2. Establish the TCP connection
  3. Execute the actual request
Steps 1 and 2 incur additional overhead, depleting resources. To mitigate this, HTTP clients maintain an open connection, preventing the need to repeat DNS resolution and TCP connection establishment. This not only saves time but also protects resources.

Node.js does not cache this by default! That's it. We need to manually set it to true. If you are focusing only on response time or opting for parallel executions, you might easily overlook this resource wastage and even end up crashing the application.

Node.js Worker Threads

Keep in mind that DNS resolution is handled by worker threads in Node.js, making it a costly blocking operation. By default, there are four threads in the pool, potentially causing Node.js threads to become busy.

Operating System Limitations

Due to non-caching connections, ephemeral port limits will be reached, preventing further outgoing calls. Note that once a port is used, the operating system will not allow its reuse for 60 seconds by default, leading to no available ports. Also, note that, due to the 60-second grace period, the actual number of available ports per second is very limited.

A note on AWS Lambda

This can be a significant issue for AWS Lambda, as for each request, a new connection is opened, resulting in extra latency. Hence, it is advisable to keep the connections open to minimize latency and prevent extra costs.

Benefits

Better performance is an obvious benefit. However, it is not the only one.
  • Increase the application's performance.
  • Prevent application crashes due to excessive resource usage.
  • Reduce the number of containers/pods needed.
  • Reduce costs (due to the above reasons).
  • Save network costs (resources, money).
  • Reduce Cloud limits (DNS calls, etc).
  • Finally... Happy developers, happy customers!
See, a simple 'true' brings you a lot of happiness.

Demo

This can be demonstrated by creating a basic Node.js application.
npm init -y
npm install axios
// npm install agentkeepalive
"type": "module", // enable ES6
// with-no.js

import axios from "axios";

const tester = async () => {
  console.log('start');
  const startTime = new Date().getTime();
  for (let i = 1; i <= 10; i++) {
    await axios.get(`https://jsonplaceholder.typicode.com/todos/${i}`);
  }
  const endTime = new Date().getTime();
  console.log(`end in ${(endTime - startTime)}`);
}

tester();
// with-yes.js

import https from "https";
import axios from "axios";
// import Agent from "agentkeepalive";

// -> It is better to use agentkeepalive
// const httpsAgent = () => {
//   return new Agent.HttpsAgent({
//     maxSockets: 100,
//     maxFreeSockets: 10,
//     timeout: 60000,
//     freeSocketTimeout: 30000,
//   });
// }

const tester = async () => {
  console.log('start');
  axios.defaults.httpsAgent = new https.Agent({ keepAlive: true });

  const startTime = new Date().getTime();
  for (let i = 1; i <= 10; i++) {
    await axios.get(`https://jsonplaceholder.typicode.com/todos/${i}`);
  }
  const endTime = new Date().getTime();
  console.log(`end in ${(endTime - startTime)}`);
}

tester();
Take a moment to compare the difference. This simple test made an approximately 6-second distinction on my laptop!

That is it ☺ Keep those connections open guys!!! Happy coding! ☺

References

https://nodejs.org/api/http.html#new-agentoptions
https://nodejs.org/api/https.html

Comments

Post a Comment