Seite wählen

NETWAYS Blog

Prometheus Webinare in 2023

Letzte Woche haben wir im Rahmen unserer Webinare auf YouTube einen groben Überblick über die Lösung Prometheus gegeben – das Video kann man sich hier ansehen, sofern man den Termin verpasst hat.

Um neben unserer Beratungsdienstleistung das Thema Prometheus weiter auszubauen, haben wir beschlossen Anfang 2023 eine Webinar-Reihe für die Lösung durchzuführen. Dasselbe haben wir bereits für Icinga und Elastic durchgeführt und für Graylog eingeplant. Ziel der Prometheus Webinar-Serie ist es, nicht nur den Aufbau selbst zu begleiten, sondern verschiedene Integrationen aufzuzeigen und die Möglichkeit, eigene Exporter zu bauen.

Webinar Themen

Zum jetzigen Stand haben wir uns folgende Themen überlegt:

  • Installation von Prometheus Komponenten mit NETWAYS Paketen
  • Anbindung externer Systeme mittels Exporter
  • Konfiguration von Alarmierungen
  • Schreiben eigener Prometheus Exporter
  • Integration von Prometheus und Icinga

Zeitplanung

Die Webinare werden im Laufe von Q1 und Q2 2023 stattfinden. Die genaue zeitliche Gestaltung wird in den nächsten Wochen erfolgen und direkt auf unserem YouTube-Kanal einsehbar sein. Am besten für eine schnelle Übersicht den Kanal abonnieren und sich bei neuen Videos und Ereignissen benachrichtigen lassen!

Sofern noch Themenwünsche offen sind oder wir generell etwas bei den Webinaren berücksichtigen sollen, am besten direkt Kontakt mit uns aufnehmen. Alternativ stehen wir natürlich beratend bei der Konzeptionierung, dem Aufbau und der Integration zur Seite und bieten eine Prometheus Schulung an – wir freuen uns über Anfragen!

Christian Stein
Christian Stein
Manager Sales

Christian kommt ursprünglich aus der Personalberatungsbranche, wo er aber schon immer auf den IT Bereich spezialisiert war. Bei NETWAYS arbeitet er als Manager Sales und berät unsere Kunden in der vertrieblichen Phase rund um das Thema Monitoring. Gemeinsam mit Georg hat er sich Mitte 2012 auch an unserem Hardware-Shop "vergangen".

OSMC 2022 | Recap Day 1

Welcome to our recap series of the Open Source Monitoring Conference 2022 (OSMC 2022) in Nuremberg. Last year the workshop and hackathon concluded the OSMC. This year three workshops will kick-off our long awaited conference. While the conference will be consecutive talks about amazing new innovations and features the workshops are the calm before the storm.

In a calm but constructive and educational environment, our experienced trainers provide theoretical insights and practical exercises in their field. Today I will guide you through the following workshops: Kubernetes, Icinga  Director and Director Branches and last but not least Prometheus.

A little disclaimer beforehand: this blog post is no detailed overview of the workshops. If you want to learn more about one of the topics I highly encourage you to visit the respective training, it’s worth it. With this said: let’s get going!

 

First Stop: Container Orchestration with Kubernetes

Our colleague in NETWAYS Professional Services Consulting Daniel Bodky is holding his first workshop at OSMC, after he got a first impression of the conference as a visitor last year. Already at the beginning he sets the stage for his Kubernetes workshop. The next few hours are planned to be a dialog with impressions from the paritcipants’ work reality.

Since Kubernetes is a ’new‘ technology for container orchestration, the group of participants is mixed. Participants with experience in the area of Docker and Kubernetes as well as beginners are present. So the starting point of the workshop is Docker. Because before you can deal with Kubernetes, it is important to learn and know about Docker containers and how they work. They are the very basis for the successful use of Kubernetes.

To build this base Daniel guides his attendees through the theoretical and practical handling of Docker images. Now that all the groundwork is done the attendees take their first steps in Kubernetes … with a theoretical overview of what Kubernetes is and isn’t. After all this theory it is time to put the gained knowledge into action. The following practical topics like Deployment, Service & Ingress or Volumes in Kubernetes offer the possibility to get hands-on experience.

Through the structure of theory (for all levels of knowledge) and practical exercises to apply the new knowledge all attendees can easily participate and achieve success. With this impressions of the OSMC Kubernetes workshop let’s go to our next stop: Prometheus.

 

Second Stop: Metric Monitoring with Prometheus

The second ‘new’ tool which got a lot of buzz over the last few years is Prometheus. Why is it so popular? Apart from the big companies using it (e.g. SoundCloud and Docker) it is using a different monitoring approach than well known tools like our Icinga 2 or Nagios.

The Prometheus-team is putting metrics and data in the center of their monitoring approach. With his vast ecosystem of different components it can be complex if getting into it without the proper introduction. The trainer of this workshop Julien Pivotto of O11y knows this from experience and is giving his attendees the needed guidance through the Prometheus world and how Prometheus and its different components work together.

As it is a workshops designed for beginners in the world of Prometheus after the introduction the first hands-on lab is the download and installation of the Prometheus ecosystem. After the first hands-on experience is done and the first fear of contact is lifted it is time to get accustomed with some more essentials. Like with every new tool it is important to understand two things: wording and how the system works. Like which functions and aggregators are commonly used in Prometheus or what tools help you use and understand the metrics of Prometheus better. Julien is showing a lot of insights in common mistakes and also the complexity of Prometheus.

Why complexity? Because Prometheus is collecting ALL THE METRICS. If there are no conditions set a lot of data is displayed. So it is important to know what you want to check and customize Prometheus to meet your goals. There are so much more topics I could write about like Grafana as go-to Dashboard or how Prometheus handles alerts if certain conditions are met. But it is time for our last workshop: Icinga Director and Director Branches.

 

Third Stop: Configuration with the Icinga Director and Director Branches

While the Prometheus workshop is about getting started with the monitoring tool, the Icinga Director workshop offers hands-on exercises and related knowledge about the Icinga 2 web configuration tool. In addition to the introduction to the Icinga Director, practical questions will be addressed from the beginning, since many of the participants already deal with Icinga 2 monitoring in their daily practice.

For many, the workshop serves as a support for explicit practical problems, where the experienced Principal Consultant of NETWAYS Professional Services Dirk Götz can offer best practices from his years of consulting experience. In order to make the best possible use of the Director’s many setting options, the workshop focuses on relevant setting options and how these can be and how they can best be implemented.

The many configuration options also show the strengths of Icinga 2 as a monitoring tool and the monitoring tool and the Director as a configuration tool: customizability.
No matter what logic is required, what specifications there are and what objectives the monitoring has everything can be set via the Icinga Director. Like every OSMC workshop this one also offers a balanced mix of theoretical input and hands-on labs to give the attendees the chance to test out their new knowledge.

After the introduction into the Icinga Director, Blerim Sheqa, Chief Product Office of the Icinga GmbH is introducing a new feature: Director Branches. This new tool is an extension for the Icinga Director and aims to provide a safe virtual working environment for development and administration teams.
The workflow of Director branches is based on git. You can create different configuration branches and in the end merge them into your productive environment. With this virtual safe space you and your teams can try out different adjustments and settings without the risk of crashing you complete monitoring setup.

Another benefit is the possibility for different teams and users to work at their own parts at the same time without getting into each others way. Another nice feature is the comment section every merge must have. Every time you check the activity log your administrators can check what changes have been deployed. With this look at a new Icinga Director feature our little tour through todays workshops comes to a close.

 

Last but not Least: A Little Peak Behind the Scenes

Even tough today the OSMC 2022 started with the three workshops the work to set-up the conference are still going. The team of NETWAYS Event Services (NES) are busy organizing the different sites to ensure everything is running smoothly the next two days.

Apart from me as todays blogger and NES there is also our marketing team around. Since 9.00 o’clock this morning they are busy taking pictures and videos, while helping at the check-in counter. So if you see some of their content in your social media feed be sure to like and subscribe.
If you post pictures of your attendance at the OSMC 2022 online don’t forget to use our official hashtag #OSMC to possibly get a repost from our official channels.

There is one feature of this years OSMC I’ve kept till the end: our new Activity Area! Here you can challenge other attendees at giant Connect four and giant Jenga as well as relax between the talks. We have one last surprise in store for you, but this one you have to see for yourself at the Open Source Monitoring Conference 2022 at the Holiday Inn Conference Hotel here in Nuremberg.

 

To get a few impressions of the first OSMC day I have prepared a slider with lots of awesome pictures of the first conference day. Enjoy! 😄

stackconf 2022 | How to Be a Good Corporate Citizen in Open Source

Dr. Dawn Foster is a unix sysadmin for VMware, did her doctorate on linux kernel development and has been following her tech career for over twenty years! Her main focus is community and open source work. In her talk she enlightened us about how to be a Good Corporate Citizen. If you would prefer to watch a recording head on over to YouTube to listen to her talk – or if you prefer to read all about it, go ahead and read on!

This is what I’ve learned from Dawn in her talk „How to Be a Good Corporate Citizen in Open Source„:

Collaboration in OSS Projects: Individuals, Companies, Communities

Intro Slide with Pictures of Dr Foster

Open source communities have a variety of different people involved.

A project has developers, a release team, localisation and translation teams, marketing, community managers, tech writers, users and lots of other people involved. All of these people are working together as one community towards the goal of a good project.

 

Balance

This community that works on the project is what makes the decisions on where the project goes, an outside corporate entity can not force them to adapt changes they don’t want – or that go against the direction of the project. As a company, you need to align your needs with the needs of the project. This is important to understand when making contributions, so you don’t put your employees in a position where they have to either do harm to the project or their employment.

Contribution Strategy and Plans

Aligning Goals

The first and most important step is to make sure that your companies and the project’s goal are in alignment. If this is the case, it will be so much easier to justify putting resources and effort into the project. It also makes it easier to make the team that works on the project understand the importance of their work.

Finding and focusing on projects is an important point. Look at your operations team and what tools they use – those might be a great fit to support. Are there development or deployment tools that are open source and you could support? These questions can help you figure out what to support, in order to make a better point to your superiors to help support those projects.

Communication

Make sure that all of your teams that work on open source projects communicate with each other, to avoid having conflicts in public open source projects. If your vision is aligned, you will have a lot fewer issues. You can even help organise meetups, provide discussion channels and events to further help foster productive discourse.

Which Projects?

Slide "Which Projects?"

After you know which projects to support, you need people to contribute. Maybe you already have people that have contributed in the past. Keep in mind that contributing to open source projects needs a different skill set than working on internal projects – they need to for example be comfortable receiving and reacting to feedback in public.

Staffing

You can also hire people that are already contributing to those projects – but you might need to be careful with that, because you do not want to get the reputation to aggressively poach contributors from projects. It requires a bit of nuance to make it known that you are hiring for a project, without coming across too strong.

Guidelines

Having guidelines and best practices ready for people to engage in open source projects. Try to find a good balance between providing help and guidance and not being too overbearing or scaring your employees away from making contributions. Help engineers understand what they want to do and why.

Measure Success

You also want to make sure you can measure outcomes and results. How do you pull that kind of data? It really depends on what you want to achieve – examples would be: for the goal „improving performance“ check the softwares performance data. For the goal „gain influence“ check your employees in meaningful positions in the project. You might also want to overmeasure a little bit, to have some extra data at hand, in case your focus shifts in the future.

Making Contributions as a Good Corporate Citizen in OSS

Slide "Getting Started"Before hopping into a new project, you might want to look around a little and understand how the community works and feel around a little. Look at the documentation, especially at the contribution docs and the code of conduct. All projects work differently and understanding how things work to not violate any community norms.

Start with small contributions and work your way up, instead of just working on a big addition to the project and just dumping them unannounced.

 Learn from Feedback

When you start participating in a project you need to expect feedback. Sometimes feedback will be kind, sometimes it will be worded a bit more harshly. What you need to do is stay focused on what changes you need to make on your contribution, stay kind and maybe have someone proofread what you write to catch any unwanted harshness in how you write your answers. Try not getting defensive and iterate on what you mean.

Work with the Community

You might want to connect with people that worked on similar areas that you are touching on, and collaborate. Get in touch with the people who run a project and discuss strategies with them, to offer better help and be more productive in the process!

Break up your work into smaller contributions to make it easier for the maintainers to work with you and to iterate through the process.

Remember that you have a lot less control over other people working on the project, unlike in a company where you are able to escalate issues to managers. Meet people where they are and be kind!

Relationships

Having good relationships with people you collaborate with makes it a lot more easy and fun to work together. Conferences and meetups are very important to solve issues when you can talk about something in person. Knowing the human being behind the other side of your screen can make a big difference! When you need to do something new, or have questions – having someone you know that can put you in the right direction is an incredibly valuable thing to have.

Upstream your Patches

When you maintain your patches internally, every time the project has an update there is a risk that someone will forget to apply them, or has to fix places that were touched by the upstream and the patch. If you get your patches in the upstream repository you will not run into those issues and you might help other people with them as well!

Maintenance Expectations

If you are adding larger features to a project’s codebase, make sure that you can help with its maintenance and have someone constantly assigned to that task. If you make additions to a project and then bail on it, you create a big workload for the maintainers, which will make you and your organisation look bad and future contributions to this or other projects will be a lot less well received.

Open Source Your Software

If you are open sourcing your projects, don’t just dump dead projects onto the internet and hope someone is going to take over. This is at best naive, and will also make your company look bad. Take care of your software, just the same way you would under a proprietary licence!
Maintaining a project with the community involved is a lot of work, but it pays off in the long run. Tend to your pull requests and issues and you will reap the hard work others have put into it.

If you have read through this all, you’ll be happy to hear that there is more content like this on this blog – or if you also enjoy a video about it, check out our YouTube channel with lots of recordings from our conferences!

Take a look at our conference website to learn more about stackconf, check out the archives and register for our newsletter to stay tuned!

Feu Mourek
Feu Mourek
Developer Advocate

Feu verbrachte seine Kindheit im schönen Steigerwald, bevor es sich aufmachte die Welt zu Erkunden. Seit September 2016 unterstützt es Icinga zunächst als Developer und seit 2020 als Developer Advocate, und NETWAYS als Git und GitLab Trainer. Seine Freizeit verbringt es hauptsächlich damit Video-, und Pen and Paper Rollenspiele zu spielen, sich Häuser zu designen (die es sich nie leisten können wird) oder ganz lässig mit seinem Cabrio durch die Gegend zu düsen.

Detector OSS IDS: How to Shellscript Your Own Little Free Intrusion Detection System

Today I’ll show you a side project I’ve been working on the past month to defend my personal systems and practice shell-scripting and forwarding logs. It is just a proof-of-concept that is work in progress. I have decided to share my project, because Open Source = Open World! You can find detector here on Github.

This small project follows 3 basic goals: a) minimal b) trustable c) modular & customizable:

  • Required Binaries for Checks: AWK, SED & GREP (en masse), Inotify-Tools, Tracee, TS, USBGuard, SocketStats, Dialog, (Nethogs)
  • Just run the ./install.sh or ./uninstall.sh
  • Comment or uncomment the execution of the scripts/modules in the central/privacy directories as you like

How it basically works:

– Runner: Create a 1) Systemd service with a timer, calling a 2) Watchdog with a timer, 3) calling a main (separating Operating Systems and module choices), 4) calling the modules

– Modules: 5) run checks 6) grep for exit codes  6) append a time-stamp 7) append a module tag (with a possible KV – filter for Logstash-Pipelines) ->> write to detecor-logfile | Optional:  9) output to Elastic (via Filebeat -> Logstash-Pipelines) 10) output to Icinga 2 (via passive-checks for more logic & free alerting)

Detector currently (2022/08/01) covers:

Dropping & tracking honeypots via inotifywait:

Tracking USBGuard:

Checking Camera & Microphone Activation:

Tracking Shells and Sub-Shells:

Tracking Established and Listening Sockets with their relevant Programs and PIDs, plus provided DNS-Servers and Wireguard:

Using Tracee from Aquasecurity with 4 cool flags: TRC-2 Anti-Debugging , TRC-6 kernel module loading, TRC-7 LD_PRELOAD, TRC-15 Hooking system calls:

Tracking Kernel-Symbol counters for changes on module export tables:

Now we can be happy, but why not send it to Elastic and do some more magic there?

Or add even more logic and alerting via Icinga 2! (All we have to do is create a template for a passive check, apply the passive check over a (Linux)-hostgroup and set up an API-User with the „actions/process-check-result“. Our icinga-pumper.sh POC Code gets automatically executed in the $central directory, and we save ourselves the Icinga 2 agent installation, while Icinga 2 authentication happens over a certificate deployed via Nextcloud or the likes. :

TrippleCross and badbpf are some very cool offensive projects with eBPF implants I’ll try to understand and study until the next blogpost. See you by then!

If you want to learn from the people that tought me to pull such a side-project off, mostly Dirk and Thomas, then come and join us!

stackconf 2022 – The Count Down is Running!

Only one week to go until it’s stackconf time!

Get new impulses and upgrade your infrastructure to what’s currently going on and coming up in the future. Think outside the box! At stackconf, you will learn how to design and build your technology stack exactly according to the needs of your business – throughout the whole lifecycle:

 

BUILDING  > CI/CD > RUNNING > MONITORING

 

stackconf Core Value – Get Together, Learn & Exchange

This social aspect is really important to us at our conference. That’s why we provide a lot of space and opportunities for discussion and exchange. Learn from other IT engineers and architects. Get feedback on your own approaches and plans for the future.

Socializing & Networking At Its Best!

The stackconf evening event will take you to the “Capitol Yard Golf Lounge”. Located in the historic Spreespeicher in the heart of Berlin this great site will provide an absolute amazing athmosphere for creating new contacts, catching up with familiar faces and engaging in extensive exchanges with other participants and expert speakers. Expect the best and be pleasantly surprised!

Not yet registered? Then it’s time to get fast & furious!

Check out the amazing speakers line up stackconf has to offer! With over 30 international infrastructure experts from top companies such as Spotify, IBM, Red Hat, Elastic, VMware, Intel, and many more.
Ceate your personal conference agenda and enjoy the great opportunity to engage with a bunch of like-minded people for 2 days of open source community feeling at its finest!

So hurry up and get your ticket now!

We are counting down the days to see you in Berlin!

Pamela Drescher
Pamela Drescher
Head of Marketing

Seit Dezember 2015 ist Pamela Anführerin des Marketing Teams. Mit ihrer stetig wachsenden Mannschaft arbeitet sie daran, NETWAYS nicht nur erfolgreicher, sondern auch immer schöner zu machen. Privat ist sie Dompteurin einer Horde von drei Kindern, zwei Pferden, drei Katzen und einem Hund. Für Langeweile bleibt also keine Zeit!