Day two of the OSMC 2023 started rather quiet, but with a interesting set of talks. The following is a summary and review of some talks I watched and was interested in. Therefore not all of the talks are mentionend here and this should not be interpreted as a judgement of their quality or significance.
Automated update management with Renovate
Sebastian Gumprich describes his journey of introducing Renovate at scale at his work place. Renovate is a software for updating dependencies in software projects, which can be self-hosted and is therefore applicable in practically every environment.
Renovate analyses the software project which is called upon, detects the dependencies, fetches data about the available versions of those and applies then updates, if any are available, and it is configured to do so.
To integrate it better into the existing development process and to not apply more load on the developers, an application as a GitLab pipeline was chosen and realized. This approach was also scalable over a huge number of different projects and repositories then.
To work correctly (and do anything) Renovate needs some configuration, which is presented as JSON and, in most cases, rather small and easy to do
The presentation was partly about the technical ideas and problem, but also, arguably more importantly, about the human part, which I found most interesting. Part of this was, unsurprisingly, structured and extensive documentation of the relevant steps and procedures and common problems. But also some programmatic features were introduced, for example, automatically opening Issues in GitLab for faulty Renovate configuration.
To further reduce the hurdles to apply Renove to a specific project, the “Onboarding” Merge Request applying the relevant changes were quite verbose in what it should do, what the consequences would be and where and whom to ask in case of open questions.
These point may seem obvious or even trivial, but, and this is the opinion of the author, organizing different people and groups of people and communicate in a constructive and efficient way is one of the biggest hurdles in the business and approaches to this set of problems are often quite interesting and helpful.
Replacing NSClient++ for Windows Monitoring
The second talk I want to advertise here is Sven Nierlein’s presentation of a replacement for the NSClient++.
The start of talk was the expectable review of the NSClient++, a monitoring agent which was quite common in different availability and status monitoring setups in the past, especially on windows operating systems. Sadly the developement is progressing slower nowadays than in the past and some problems, which were not fixed, are increasingly a dealbreaker. Especially, some problems with the lack of current TLS protocols are problematic.
Writing a new agent was not really the first choice, but a comparison of current alternatives did not present a good solution since the introduction of completely new configuration, new protocols and different workflows was not a feasible way to go. The resolution was therefore to write a completely new, but compatible monitoring agent.
This offered some freedoms regarding the choice of tools. The choicethen went in the direction of the Golang language and the related toolchain. The new agent was called SNClient+ (where SN stands forSecure Naemon) and supports multiple protocols from the side of themonitoring system.
One of the is the NRPE protocol for compatibility reasons and, the prefered method, an HTTP-based method, which can be used with chec_nsc_web.
Additionally, to add more features, a general Prometheus exporter wasintegrated, which exposes the general operating system exporters of thePrometheus ecosystem. Therefore, the SNClient+ can also be used as the default node exporter.
To stay compatible and enhance the functionality further, there are not only built in plugins to test different properties on the host machine, but a generic functionality to execute third-party plugins is included.
A self-updating functionality is also built-in to make updates as easy as possible.
In summary, this is a promising new solution for an old problem and is likely worth a try.
Running the Infra at FOSDEM
Rather spontaneously, Sebastian Schubert made a presentation about the infrastructure at FOSDEM, one of the largest Free and Open Source Software events in the world. The event occurs yearly at the beginning of February in Brussels, and they expect around 10.000 visitors/day with around 20.000 devices which need to be connected to the internet. This would be, by itself, a challenging task, but it is a totally different scenario to deploy that kind of infrastructure for just a few days and there are no paid professionals, just volunteers which might turn up with no idea what, where and how.
The astonishing fact, that this kind of organization actually works (and that repeatedly and successfully) can probably not be admired enough.
Additional to providing network access (and some services there), there is also the video and streaming setup for the hundreds of different talks, which must not only be recorded, but also, ideally, be live-streamed to the internet (currently over third parties).
For this purpose, self-designed hardware boxes were used in the past to re-encode the video and audio in first step on site, which are increasingly replaced by more common laptops. These serves as a kind of “render farm” to prepare the material for the viewer.
Following that was a short introduction to the tools used in the network setup and especially some problems regarding using IPv6-only network in the 2020s where some parts of the internet are still only reachable via IPv4. One example here was the usage coreDNS as a replacement for bind9 (for resource usage reasons).
A generally good idea mentioned then was the introduction of monitoring on- and off-site where data was replicated and still available when there was an incident which took the equipment of the FOSDEM crew at the university offline.
Another interesting point added was the general availability of practically all relevant material to the, public which allows interested parties to get some ideas how everything works there and maybe allows the adaption to other purposes.
openITCOCKPIT Community Edition – Einfache Konfiguration, Module, API und mehr
In this talk, Jens Michelsons presented openITCOCKPIT monitoring system, which is one of the “Nagios-similar” monitoring systems they created at the it-novum company.
The focus lies there on creating an easily usable web-based system, where everything is integrated. A powerful HTTP API serves as the main interface for all the different components and is well documented. This allows small scale configurations via the web interface or more automated setups with other tools.
A speciality of openITCockpit is problably their own monitoring agent for remote hosts and the strong integration of other tools, including the CheckMK agent, into their systems. A migration of an existing setup in openITCockpit or extending one with other tools is therefore less painful than it could be.
Remarkable was also the extended live demo (always a risk in a presentation) which presented a typical but not simple workflow for adding some systems to the monitoring, including a combining logic of different tests.
Zabbix – Powerful enterprise grade monitoring driven by Open Source
Appropriately, the following talk was about Zabbix, a system quite similar in many regards to openITCockpit. Wolfgang Alper described the working principles of Zabbix and what the main concepts and functionalities are.
The direct comparison was quite interesting, as one can recognize common ideas and components, but also where philosophies and ideas differ and how different problems were addressed.
One of the most important ideas in Zabbix is the separation of concerns, where gathering of data, storage, problem detection, alarming and escalation are split up programmatically and can be treated individually. The definition of these steps and their interfaces allows developers to focus on a specific part without having to worry about the whole.
Another part of the talk was dedicated to how Zabbix handles large scale and distributed setups. At this point, a part of the Zabbix software components which is called “Proxy” comes into play, and relays directions from the central system to outliers and data the other way round.
All in all Zabbix is probably a capable tool to do the classic network monitoring task, but of course not limited to that.
