Select Page

OSMC 2022 | What’s new in the Prometheus ecosystem?

by | Dec 13, 2022 | Events, OSMC

About a month ago OSMC took place and I was happy to attend an offline conference again. One of the things I enjoy most at conferences is networking and this is why I know Julien Pivotto already for years. He focused his work a while ago on Prometheus and is a regular speaker at OSMC, typically not only giving a talk but also a workshop. And now he is also a maintainer of Prometheus. So who could be a better speaker to give an update talk summarizing the news from the last year?

Julien started with a short introduction to Prometheus which I want to repeat here as some readers will also likely not be familiar with the tool. Prometheus is an Open Source monitoring solution based on metrics. It provides a Service Discovery to find metric endpoints to pull data from. PromQL – the Prometheus Query Language – is then used to query and visualize data in the frontend. The project while sometimes still called a new competitor in the field of monitoring solutions has already some history and is a graduated CNCF project since 2018, so we do talk about a project that already left its footprints in the monitoring ecosystem.

Helpful when Exploring Metrics

The first topic Julien covered was the new react UI providing a query editor for PromQL including autocompletion and formatting. The metrics explorer now also supports using the local timezone instead of UTC which can be quite helpful when debugging as it removes the need to calculate this. Also very helpful is the option to filter targets and service discovery to reduce the amount of shown data. The option to show and explain CLI flags helps to tailor your installation to your needs. And as every software it now has to provide also a dark mode!

The next feature he introduced was the agent mode allowing to have an instance which only sends metrics and does not provide all the overhead like the UI if not needed. I would say the remote write receiver is a corresponding feature as it allows for federation of writes by rules as another system tailored to your needs.

The list of service discoveries increased by many new cloud and configuration management solutions, but the one to highlight is the Generic HTTP which allows for the most common and easy to implement one, a REST API with JSON body.

Relabeling capabilities are quite nice and simple ones like uppercase or lowercase are a common use case. But setting some specific labels also allows to do some scraping configuration like changing the interval.

Additions to PromQL

The additions to PromQL were a bigger part of Julien’s demo. One of the many new functions to mention is last_over_time which fills the graph with the last value coming in handy if a target does not provide metrics regularly. The option to have a negative offset is also great to mitigate errors in this case if a system was not in sync and did send with an offset of some minutes. But the most important new function is the new modifiers allowing you to get for example the top 5 at a specific time and then get the values for this specific metrics. This allows to visualize how the top 5 developed over time instead of simply showing the top 5 without how they have changed over time. By doing so you get much nicer and informative graphs.

Julien Pivotto doing his talk

Out of time metrics support allows to fill old metrics in the configured interval, so you can allow this temporarily to fill in gaps if needed by adding a timestamp to the metric.

The new LTS version allows the users to have 6+ month stable APIs while the developers can improve and release still every 6 weeks. Julien showed also how prominent they show the LTS version on the download page and documentation. This is something other projects could learn from. Talking about versioning they also synchronized the end user and go module’s minor version to show the compatibility.

Only the Required Features Set

The developers also introduced a plugin system which allows to build Prometheus with only the required feature set like only the needed service discoveries to reduce the size of the binary.

Also the Alertmanager got some new features. There is the option to mute alerts based on time for receivers allowing for example to honor on-duty and holidays. Negative matchers allow for a simpler ruleset and the new receivers provide more ways of notification.

Exporters in general got systemd socket activation and can now bind to multiple addresses. The node exporter providing information about a system got more collectors and the ones for MySQL and PostgreSQL can now handle multiple targets allowing to have only one for multiple databases reducing the amount of installations.

The Prometheus ecosystem also got a new Rust client , 30+ repositories in the prometheus-community organization, a certification program to prove your Prometheus skills and a conformance program for vendors to show compatibility of their software.

Introducing PromLens

And last but not least Julien introduced PromLens, a new separate Querybuilder with a great Explain functionality detailing how your query functioned and how many results are found. While planned as a separate tool a subset of the features should also be integrated in the normal UI to improve the user experience.

PromLens (taken from https://promlens.com/)

As Julien said, it was a great amount of topics to cover and I tried to summarize further. Please watch the recording of his talk in our OSMC Archives for more details. You will get some practical examples when he demos most of the topics mentioned above as he did a great job. If you are new to Prometheus we also have you covered with our NETWAYS training course “Prometheus Fundamentals”. Check it out!

We hope to see you around at OSMC 2023! Stay in touch and subscribe to our Newsletter!

Dirk Götz
Dirk Götz
Principal Consultant

Dirk ist Red Hat Spezialist und arbeitet bei NETWAYS im Bereich Consulting für Icinga, Puppet, Ansible, Foreman und andere Systems-Management-Lösungen. Früher war er bei einem Träger der gesetzlichen Rentenversicherung als Senior Administrator beschäftigt und auch für die Ausbildung der Azubis verantwortlich wie nun bei NETWAYS.


Submit a Comment

Your email address will not be published. Required fields are marked *

More posts on the topic Events | OSMC

stackconf 2023 | Recap Day 2

After a fun evening event filled with drinks, socializing, and even some roulette, hopefully everyone got some rest. Once that first cup of coffee kicked in, the attendees were ready to dive into the second day's talks at stackconf 2023.   Let the Talks get...

stackconf 2023 | Excitement is in the Air!

Are you ready to dive headfirst into the world of cutting-edge cloud native infrastructure solutions? The wait is finally over as the highly anticipated stackconf 2023 is set to begin tomorrow! Hello Berlin! Yesterday, Katja and Lukas took their packed car and drove...

OSMC 2023 | Dive into our Conference Timetable

As OSMC 2023 draws nearer, we are thrilled to reveal that the conference program has been finalized! Anticipate two days filled with a grand total of 34 lecture sessions conducted by renowned experts in the open source field. This year’s edition of the event is...