Enabling Prometheus w/ labels from Vert.X metrics

Vert.X uses Dropwizard metrics under the hood, by default, when metrics are enabled (the other option being Hawkular Metrics). They can be exposed as Prometheus endpoints, as described here. It works without any pain, but has a downside: the metrics don’t make any use of Prometheus labels. Which can be annoying when you want to build dashboards or run elaborated queries.

A reason for that is Dropwizard not handling tags or labels yet. A PR exists and is merged, but not in the current release branch. It will be in the long-awaited version 4 of Dropwizard metrics.

But there’s a quite simple workaround to solve this issue: using Prometheus’ metric relabel in the scraper configuration. Here’s a short example that will translate a metric vertx_http_servers_0_0_0_0:8081_responses_2xx_total to vertx_http_servers_responses_total{code=2xx,server=0_0_0_0:8081}.

As you can see, extracting a label from the metric name takes two steps:

  • The first step is to locate and extract the desired value using a regexp pattern and assign it to a label name.
  • The second step is to rename the metric by removing that value from the name, using the exact same regexp pattern.

This two-steps operation is repeated for each label we want to extract.

Note that __name__ refers to the metric name, which can be seen as a reserved label.

This entry was posted in Monitoring. Bookmark the permalink.

Leave a Reply