How to collect metrics from node.js applications in PM2 with exporting to Prometheus

It’s no secret that for stable and reliable working of node.js applications, it’s necessary to monitor their performance and get useful insights from their metrics. It means being able to collect information about app state before issues appear to preventing failures.

In this article, I’d like to present a method for collecting metrics node.js applications from running in PM2 and exporting this data to Prometheus.

When you simply start a node.js application using the node command or through PM2 with the pm2 start app.js command the new application is launched as a single instance. Collecting and exporting any metrics in this case would not be difficult. To do this, you can install the prom-client package and add the metrics you need, for example, the number of requests to our application.

import client from 'prom-client';
import { createServer } from 'http';

const registry = new client.Registry();
const PREFIX = `nodejs_app_`;

export const metricRequestsTotal = new client.Counter({
    name: `${PREFIX}request_counter`,
    help: 'Show total request count',
    registers: [registry],
});

// Start your application
// ...
nodejsapp.get('/*', async (req, res) => {
    metricRequestsTotal.inc();
});

// Start Prom server for export metrics
const promServer = createServer(async (req, res) => {
    res.setHeader('Content-Type', registry.contentType);
    res.end(await registry.metrics());
    return;
});

promServer.listen(9100, () =>
    console.log('Prom server started')
);

In this example, when your application is launched, it also stars metric exporting on an additional port 9100. Of course you can use a separate URL with the main application, but it shouldn’t be public for users.

This approach will work as long as you don’t need to run multiple instances of the application in cluster mode. For collecting metrics in cluster mode, prom-client provides a good example with metric aggregation. But what if you’re running the application through the process manager PM2, where you don’t have access to the running cluster?

In one of my previous articles, I talked about the pm2-prom-module, which allows collecting overall statistics for PM2 applications but couldn’t collect internal statistics from each application. However, starting from version 2.0, such limitation is no longer present, and all statistics from PM2 and within the application are provided through a single module.

To communicate between node.js application and pm2-prom-module, you need to install the npm package pm2-prom-module-client in your application. As a result, the example above will look like this:

import client from 'prom-client';
import { initMetrics } from 'pm2-prom-module-client';

const registry = new client.Registry();
const PREFIX = `nodejs_app_`;

const metricRequestCounter = new client.Counter({
    name: `${PREFIX}request_counter`,
    help: 'Show total request count',
    registers: [registry],
});

// Register your `registry` to send data to pm2-prom-module
initMetrics(registry);

// ...
nodejsapp.get('/*', async (req, res) => {
    // ...
    metricRequestCounter?.inc();
    // ...
});

Now pm2-prom-module will provide PM2 statistics and also internal statistics from your applications.

/// Common metrics from PM2

# HELP pm2_free_memory Show available host free memory
# TYPE pm2_free_memory gauge
pm2_free_memory{serviceName="my-app"} 377147392

# HELP pm2_cpu_count Show available CPUs count
# TYPE pm2_cpu_count gauge
pm2_cpu_count{serviceName="my-app"} 4

# HELP pm2_available_apps Show available apps to monitor
# TYPE pm2_available_apps gauge
pm2_available_apps{serviceName="my-app"} 2

/// Apps metrics
# HELP nodejs_app_request_counter Show total request count
# TYPE nodejs_app_request_counter counter
nodejs_app_request_counter{app="app1",serviceName="my-app"} 10
nodejs_app_request_counter{app="app2",serviceName="my-app"} 17

By default, all statistics are aggregated across all running instances in every app, but if you want detailed statistics for each instance, you can enable this through the module’s configuration parameter:

pm2 set pm2-prom-module:aggregate_app_metrics false

This module, unlike some others, allows collecting metrics separately for each application, and they do not intersect.

Overall, the statistics provided by PM2 are enough for basic application monitoring. However, in our case, for example, we wanted more detailed information about page rendering time or loading of certain data blocks. For such purposes, we used Histogram metric type.

// ...
export const metricRequestTime = new client.Histogram({
    name: 'nodejs_app_page_execute',
    help: 'Time to processing request',
    registers: [registry],
    buckets: [0.1, 0.2, 0.3, 0.5, 0.7, 1, 2, 3, 5],
    labelNames: ['code', 'page'],
});

// ...
const endMetricRequestTime = metricRequestTime.startTimer();
// ...
endMetricRequestTime({ code: 200, page: 'Homepage });

In example above, we use 2 additional parameters: code(HTTP response code) and page(page identifier), which provide detailed statistics. For example, you can build such graphs in Grafana:

• Most frequently pages sum(rate(nodejs_app_page_execute_count{serviceName=”$serviceName”,code=”200"}[5m])) by (page)
• The number of HTTP (response codes) success or errors. sum(rate(nodejs_app_page_execute_count{serviceName=”$serviceName”}[5m])) by (code)
• The page load time specifying, for example, the 99th percentile. histogram_quantile(0.99, sum(rate(nodejs_app_page_execute_bucket{serviceName=”$serviceName”, code=”200"}[1m])) by (page, le))
• And many other graphs for each individual page.

By the way, I don’t recommend using the URL of the page as the value for page — it will significantly increase memory usage and the size of the response in Prometheus, and mess in the graphs. It’s better to use page identifier.

So, to summarize, we have a single point for collecting and serving statistics for all applications running in PM2, and a monitoring system built on Prometheus with functional dashboards in Grafana.

Grafana charts from pm2-prom-module data

If you like pm2-prom-module, I would appreciate if you could star it on Github. Thanks for reading.

UPD: Link to my grafana dashboard with pages and codes