Wie bekomme ich bessere Informationen über Züge als die DB

marudor.de screenshotAls reisender Consultant mit Bahncard 100 verbringe ich sehr viel Zeit in Zügen der Deutschen Bahn. Das ist auf der einen Seite erfreulich, denn ich verbringe die Zeit nicht auf der Autobahn. Auf der anderen Seite macht es die Bahn einem manchmal schwer, sinnvoll von A nach B zu kommen. Es soll ja schon einmal vorgekommen sein, dass Züge ausfallen oder zu spät kommen. Möglicherweise ist auch mal ein Zug überfüllt, Klima defekt, Bier alle oder sonst etwas.

Die DB selber bietet für so etwas ja seit einiger Zeit den DB Navigator. Die wichtigsten Daten bekommt man hier auch. Allerdings weiß die Bahn eigentlich über ihre Züge viel mehr als sie uns sagt. Dieses Problem löst sehr gut und schon seit einiger Zeit die Seite marudor.de

Man sieht auf der Seite, wenn man nach einem Bahnhof, einem Zug oder einer Route sucht, ein gute Übersicht über den gewünschten Zug. Hier sieht man auf einen Blick:

  • Reihenfolge der Waggons mit EXAKTER Positon am Bahnsteig
  • ICE Baureihe inkl. Revision (dadurch weiß man auch, ob man alte bequeme oder neue unbequeme Sitze bekommt)
  • Die Postition der Comfortsitze im Zug, so dass man direkt da einsteigen kann (roter Punkt)
  • Die Position von Bordrestaurant, Kinderabteil, Behindertenbereich und -toilette
  • Ruhe und Handy-abteil
  • Wifi Accespoints (und ob er funktioniert)
  • Aktuelle UND vergangene Störungen auf der Fahrt

Teilweise bekommt man diese Infos auch woanders, allerdings sind sie bei marudor meistens aktueller und vor allem alles in einem Rutsch. Und nicht zuletzt: Verspätungen und Verspätungsprognosen sind besser und zuverlässiger.

Warum das alles so ist und wieso der marudor das alles macht kann man hier erfahren. Das Video kommt von der GPN 19 und ist sehr interessant anzuschauen.

Christoph Niemann
Christoph Niemann
Senior Consultant

Christoph hat bei uns im Bereich Managed Service begonnen und sich dort intensiv mit dem internen Monitoring auseinandergesetzt. Seit 2011 ist er nun im Consulting aktiv und unterstützt unsere Kunden vor Ort bei größeren Monitoring-Projekten und PERL-Developer-Hells.

Open Source Camp on Foreman

Like every year there was an Open Source Camp following the OSMC and as usual we helped organize that. Just in case you aren’t aware of what an Open Source Camp is here is the just of it: It’s meant to be an offer for Open Source projects to present themselves more in depth to the community. This year the Open Source Camp is on that one special yellow helmet we all know and love, Foreman.

Ondřej Ezr started us off with Ansible automation for Foreman (hosts). There are probably more than enough people using puppet only in their Foreman environment. Alternative or complementary to that would be using the plugin foreman_ansible. Ansible and Puppet don’t necessarily need to be better or worse, they are different and both have their advantages and disadvantages. By going through some basic steps, like role assignment, host creation and so on, he showed how one can do all that, but with Ansible. You can easily dynamically allocate roles and installations through Ansible to your Foreman hosts, but to make it even more specific one can set custom variables within the Ansible plugin for it to use, like foreman_repository_version. You could invoke a Job, like an Ansible Playbook, which will overwrite the variables previously set or make your installation more customizable from the get go. Install from git, run a playbook through ssh and more was covered during his talk. The plugin would not be a good alternative or viable if it did not hold up against the standards that puppet sets as a competitor. While Ansible doesn’t offer an inherit solution for reoccurring runs like every hour, the plugin does.

Next up was Bernhard Suttner, who wanted to give us a taste of Salted Foreman. Initially he explained what all that salt was about. The SaltStack a open source project written in python, can be used as a configuration management tool for Foreman. Salt excels at orchestrating cloud environments and network use-cases, but then we got to the Foreman relation. Running a salt and Foreman environment means running a environment of managed hosts, which are salt minions and a foreman_smart_proxy, which will also be the salt master. He showed us what salt in Foreman looks like and gave us some insight on how it works, but even more important from now on there are people dedicated to the project and some day the plugin might be as good as the puppet or ansible plugin. Salt is great and especially effective in terms of scalability. It’s pretty straightforward to use and the initial setup is not so hard. We are excited for what is to come.

Provisioning on Azure Cloud through Foreman by Aditi Puntambekar was going to follow that one. Aditi made sure everyone is familiar with the extend of Foremans capabilities in terms of provisioning. This was especially important because Foremans capabilities differ from its usual when it comes to cloud provisioning. After a quick trip through the configuration of compute resources and imaged-based provisioning templates we went onward to the Azure Resource Manager. She explained how the Azure Resource Manager essentially worked, but what is interesting to us is the foreman_azure_rm. Well and foreman_azure_rm does what you expect it to do. It adds the Microsoft Azure Resource Manager as a compute resource for the foreman. In her demo, she showed us how to use said resource and more.

Martin Bačovský talked about CLI tools with Foreman. He started of with the Foreman API. Of course the Foreman API is fast and has a wide range of tools and libs included within it. Just like Martin said in his talk, if you are interested in the Foreman API check out the documentation, it’s very good. Also interesting in the realm of APIs was his next tool, which is using apipie/apipy, which you are probably aware of if you are more heavy on the python side of things. Up there with the most well-known tools is Martins next, Hammer CLI, a command-line tool for Foreman. After sharing his experience with these rather popular tools with everyone he introduced us to Foreman’s integration of GraphQL. It’s basically a query language, which seems to be promising so far. Martin especially focused on the flexibility of queries and the introspective it has, yet one has to see where the project goes. There were many more tools he told us a lot about. To name just a few more of them, Report Templates, Foreman Ansible Modules and foreman_maintain. If you are interested in one of these tools in particular check out the video of the talk, which will be available soon on our Youtube Channel.

 

Give your Foreman a greater toolbox with Plugins by our very own Dirk Götz. Like he said himself: I will start of with existing toolbox things and at the end I will show you how to create these things yourself. And that he did. This talk was very demo heavy, thereby everything he explained was plain and simple, because you where able to see it as he did it. At the very top of his agenda was Job Invocation/Remote Execution. Not that exciting you think? Well, more interesting is the best practice advice he threw in on the way, like there is no issue of the configured user because his password is not saved as plain text in the database. Then the development part was up. He showed a couple of jobs that he wrote himself. Easiest, which served as an example is a simple ping check. He pointed out important thoughts to keep in mind, while writing jobs, like default values. Before his talk came to a close he talked a bit about the Web Console which has been introduced and is yet not well known. The web console is pretty much a integration of Cockpit. A well experienced user in the Linux world won’t be that excited about this, but a less experienced user will love this.

The next talk would not have happened, if Dirk didn’t spontaneously offer to step in. So we got another thirty minutes of Dirk Götz and I won’t complain. Katello: Adding content management to Foreman was the title and people where keen to hear about just that. What is Katello? Dirk described it as a defined set of Foreman plugins but not just that. It enriches your content management, as well as subscription management. Wait… content management? Why do I need that? Configuration management should be enough! Not necessarily, depending on your environment. Lets just pick up the points that Dirk made towards content management. For local content it ensures availability. For staging, it allows testing updates and makes builds reproducible. So content management should be seen as an addition to config management. He also talks about content views and how they are used to do the versioning, while they are being held by life cycles. Integration in orchestration was also a rather big point during his talk, which is done via SSH or Ansible. Dirk designs his talk in a way that makes summarizing them impossible, because he covers way to much. Lets just say not announced but very appreciated and most definitely worth checking out at our NETWAYS-Youtube Channel.

It was my second Open Source Camp and if you ask me this kind of exchange is what one wants to see in the open source community. There was variety and judging by the crowd reactions I was not the only one enjoying these talks. Thanks to all the speakers and attendees, safe travels home to everyone. Until the next Open Source Camp, hope to see you there!

Alexander Stoll
Alexander Stoll
Junior Consultant

Alexander ist ein Organisationstalent und außerdem seit Kurzem Azubi im Professional Services. Wenn er nicht bei NETWAYS ist, sieht sein Tagesablauf so aus: Montag, Dienstag, Mittwoch Sport - Donnerstag Pen and Paper und ein Wochenende ohne Pläne. Den Sportteil lässt er gern auch mal ausfallen.

OSMC 2019 – Day 2

OSMC Logo
The social event yesterday evening was a blast and the late lounge afterwards also a must. So while some were still recovering, the room for the first talk was already quite full. This showed the interest in Jochen Kressin‘s talk about “Zero Trusted Networks – why Perimeter Security is dead”. He explained the (old) assumption of perimeter security “I am behind a firewall, so my traffic is secure” and asked the question if this is still true. Showing examples proving it is not true anymore because if it would be, none of these data breaches would have been happen. He explained what has changed in the last years leading to “Zero Trusted Networks” where every system has to be treated as untrusted and how to adopt for it. As one of the developers he used Search Guard as example which adds security to Elasticsearch, one of the great tools that had no security by itself for a long time, being not ready for the zero trusted approach.
Zero Trusted NetworksFluentD
Toshaan Bravani was talking about “Monitoring your Logs with Fluent”. FluentD and the client component FluentBit is an alternative to Logstash I see more and more at customer environments, so I was happy to get a deeper look into it. In addition he showed the complete tool stack to get most out of your data and the automation used to get it up and running.

Open Source landscape for APM
Third one for today was “Improved Observability Using Automated, OpenCensus-based Application Monitoring Solutions” by Tobias Angerstein. He started with a nice overview of the Open Source landscape for Application Performance Management before focusing on inspectIT. Its latest incarnation inspectIT Ocelot focuses on Open Standards like Open Metrics, Open Tracing and Open Census which are forming a new one called Open Telemetry which allows integration with all the well-known tools like Telegraf, Prometheus, Grafana. It also provides End User Monitoring using Boomerang, a javascript agent, and an EUM Server which transforms data to the Open Standards. In his demo he showed the capability of it and my only thought was how helpful something like this would have been to me in my early IT days being a Java developer.

Afterwards we could enjoy another great lunch break and perhaps also an massage, before starting into the afternoon sessions.
Lunch breakLunch BreakLunch Break

Database observability
Charles Judith gave a talk about “How to improve database Observability”. In his job he is responsible for reliability of the company’s databases and told the crowd the problems he started with like having no backup and monitoring at all. So it was his personal goal to have no hidden issues anymore and get transparency into their environment. His way from zero to hero was quite interesting and he compared it with a roller coaster. In the end having metrics to tell users that they are right or wrong with their feeling of the database is slow and having logs and monitoring telling were the real problem lies instead of guessing has improved his daily work already. But he still has some more steps to do like publishing SLA. The WIP version of his toolkit can be found on Github.

Why BOFH is toxic
Second last one I attended was Jan Doberstein with a non technical talk about behaviour and how it influences your daily life and work, titled “Idiot! – or: Why BOFH is toxic”. He touched the same topic like the open discussion yesterday and I think it is great to get people think about and reflect their behaviour. While most of his examples were matched to the crowd and perhaps people working in IT do communicate much more in electronic fashion than others, it is a topic that everyone should care about.

High available setup
Last but not least Marcel Weinberg showed the high available setup he built for Digital Ocean. He included some very helpful small tips and tricks to increase performance and avoid pitfalls while diving deep into the configuration. Indeed it were too much for me to list them all here.

Pictures are taken again from the OSMC stream at Twitter, thanks to everyone for sharing their impressions. I hope everyone enjoyed the conference like I did. Thanks to everyone who made OSMC such a great experience again this year, starting with my colleagues organizing the event, the sponsors and speakers but this includes every attendee forming this nice community. Save travels for everyone leaving today or see you tomorrow if you join the Hackathon or Open Source Camp on Foreman. I hope I will see everyone next year at the same place on November 16th to 19th for OSMC 2020 or in Amsterdam for IcingaConf on May 12th to 14th.

Dirk Götz
Dirk Götz
Principal Consultant

Dirk ist Red Hat Spezialist und arbeitet bei NETWAYS im Bereich Consulting für Icinga, Puppet, Ansible, Foreman und andere Systems-Management-Lösungen. Früher war er bei einem Träger der gesetzlichen Rentenversicherung als Senior Administrator beschäftigt und auch für die Ausbildung der Azubis verantwortlich wie nun bei NETWAYS.
OpenStack made easy – Sicherheitsgruppen verwalten und zuweisen

OpenStack made easy – Sicherheitsgruppen verwalten und zuweisen

This entry is part 3 of 3 in the series OpenStack made easy

Nachdem man sich in unserer OpenStack-Weboberfläche die erste neue Instanz zurecht geklickt und dabei einen SSH-Public-Key, mit dem man sich auf diese VM verbinden möchte, zugewiesen hat, steht der/die frisch gebackene AdministratorIn vor dem kleinen Problem, dass er/sie von außen nicht auf die Instanz kommt; das “verdanken” wir der “default”-Sicherheitsgruppe.

Sie beinhaltet die Regeln:


– Erlaube  eingehende Verbindungen mit jedem Protokoll, auf jedem Port, aber nur von Hosts im internen Netzwerk, die auch die “default”-Sicherheitsgruppe nutzen (IPv4 und IPv6)
– Erlaube ausgehende Verbindungen mit jedem Protokoll, auf jedem Port und nach überallhin (IPv4 und IPv6)

Auf diese Weise wird der Schutz der neuen VM sichergestellt. Jeder Verbindende von außen kommt nur wirklich durch die Zugangsöffnung, die dafür vorgesehen und geschaffen wurde. Um eine solche zu kreieren gibt es zwei Wege: Es kann eine neue Sicherheitsgruppe angelegt und mit einer Regel versehen oder die default-Sicherheitsgruppe um eine Regel ergänzt werden. Zweites bietet den Nachteil, dass die einzugebende Regel künftig für alle neuen Instanzen mit der default-Sicherheitsgruppe angewandt wird, was nicht immer auf allen VMs sinnvoll sein wird.

 > Neue Sicherheitsgruppe erstellen

Man klicke: Netzwerk > Sicherheitsgruppen > “+ Sicherheitsgruppe erstellen”.

Ein Dialogfeld erscheint, in dem, zwar nach Gusto, jedoch obligatorisch ein Name eingegeben werden muss (und optional eine Beschreibung eingegeben werden kann). Hier nenne ich die neue Gruppe “Beispiel”, aber jeder andere Name, der z. B. eigene Gruppierungsstrategien verfolgt, wird es tun. Dann noch Sicherheitsgruppe erstellen.

Dann erscheint in der Liste:

 > SSH-Erreichbarkeit von extern als Regel einer Sicherheitsgruppe hinzufügen

Man mause: Netzwerk > Sicherheitsgruppen > Regeln verwalten (bei der Sicherheitsgruppe, die editiert werden soll).

In einer neuen, noch unbearbeiteten Sicherheitsgruppe wird man nur jeweils eine Regel zum Austritt (IPv4 und IPv6) finden. Weiter geht es mit: “+ Regel hinzufügen”. Hier wähle man im Drop-down-Menü Regel den Unterpunkt SSH und “Hinzufügen”.

– Wenn eine bereits der VM zugewiesene Sicherheitsgruppe (z. B. default) mit dieser Regel versehen wurde, findet die Regel sofort Anwendung und die VM kann über die CLI kontaktiert werden.

– Falls eine Regel in einer neuen Sicherheitsgruppe erstellt wurde, die noch nicht der VM zugewiesen wurde:

 > Der VM eine neue Sicherheitsgruppe zuweisen

Navigiere: Compute > Instanzen > Drop-down-Pfeil (ganz rechts neben der Instanz, die modifiziert werden soll) > Sicherheitsgruppen bearbeiten. Unter “Alle Sicherheitsgruppen” findet sich die neue, mit dem weiß-auf-blauen Plus füge man die neue Sicherheitsgruppe den “Instanz-Sicherheitsgruppen” hinzu und “Speichern”.

 > ICMP-Erreichbarkeit von extern als Regel erstellen

Netzwerk > Sicherheitsgruppen > Regeln verwalten (bei der Sicherheitsgruppe, die editiert werden soll) > “+ Regel hinzufügen” > Regel = “Alle ICMP” > Hinzufügen.

 > Genauso funktioniert auch z. B. eine Regel für HTTP / HTTPS oder die folgenden

 > Regelbeispiel mit mehr Schikanen
 > Erreichbarkeit von extern mit TCP im Portbereich 65530-65535 nur von IP 200.135.41.125 aus

Netzwerk > Sicherheitsgruppen > Regeln verwalten (bei der Sicherheitsgruppe, die editiert werden soll) > “+ Regel hinzufügen” > Regel = “Angepasste TCP-Regel > Port öffnen = Port-Bereich >
“Von-Port” = 65530 > “Bis-Port” = 65535 > CIDR = 200.135.41.125/32 > “Hinzufügen”

Für alle, denen das Aufsetzen und Konfigurieren neuer VMs zu umfangreich oder schwierig erscheint, übernimmt gerne MyEngineer das Erstellen jedes gewünschten Setups.

Das erste Projekt in unserer NWS-Cloud kann hier gestartet werden.

Wie man sich die erste Maschine aufsetzt, ist in diesem Artikel beschrieben.

OSMC 2019 – Day 1

OSMC Logo
As always OSMC started with the workshop day. This time the topics were Prometheus given by Julian Pivotto, Gitlab by Michael, Terraform by Lennart and Foreman by me. After the workshops with coming together for dinner and some drink social networking started, one of things I enjoy most at conferences nowadays.

Day one started also like every time with a warm welcome from Bernd. Quite unexpected for me was the high number of first time attendees who raised hands when Bernd asked. It is always great to see new faces!

Ansible module planned to be released
First talk I attended was “Directing the Director” by Martin Schurz who gave some insights in how the monitoring platform developed at T-Systems Multimedia Solutions GmbH over last years. So they scaled up from single system, solved migration from VMs to Docker of the monitored environment and built knowledge to provide consulting to the teams which run 94 different projects which have to be monitored. With so many different things to monitor the next step of course was automation where all the good from Icinga-Director-API and Ansible came together. But if it is easy for users to build monitoring objects, configuration will grow which comes with the next challenges to make it even more easy and error prove. And from what Martin showed they solved it in a good fashion, but future will tell and I hope he will give another talk providing an update in the future.

Crowded room at OSMC
Second one was Christian with “Windows: One Framework to Monitor them all” who introduced his precious to the crowed. If you could not make it into the crowded room or was not at OSMC at all, have a look at the documentation or the framework itself, the plugins, the kickstart script to get it up and running the background daemon. In a great live demo he showed all the components and explained them in depth. While it is still the first release candidate it looks very promising.

Marcelo Perazolo from IBM Systems was talking about “Monitoring Alerts and Metrics on Large Power Systems Clusters”. He started with an introduction to the Power architecture and the workloads it is specially useful to make everyone familiar with it. The example he used was a big one, the Summit supercomputer. The main topic were two projects CRASSD and Power-Ops which not bring the data from some systems not everyone is familiar with to commonly used tools but also include Ansible playbooks for automation and flexibility “instead of just providing a docker container”. The demo showed some Kibana dashboards which provided in-depth data summarizing the health and performance of Power Systems starting from firmware to service running on it.
Power architecture explainedDesert

Lunch was great as always (and not only the dessert) and to avoid food coma we had the first time ignite talks at OSMC. It started with one from the conference sponsor ilert. Afterwards Blerim talked about “How Observability is not killing Monitoring” where he concluded that Observability should be an addition to Monitoring and not a replacement. Toshaan Bharvani decided for an ignite when sitting in the talk about Power Systems and wanted to add about “Building your own Datacenter” based on OpenPower and software available for it. In his talk “Overengineering your personal website” Bram Vogelaar showed a good (and funny) example where adding things to your infrastructure can escalate.

New entry in the datalist
Marianne Spiller‘s talk “Lorem Icinga puppetdb director amet” was not only a must see because its creative title, but because I like the mix of humor and technical knowledge she always provides. With practical examples she showed the problems of manual work like lazy admins prefer introducing a “Not answered” to the datalist of operating systems instead of maintaining the information on hosts. So instead of a form manually filled by admins she ended using the import from PuppetDB to create monitoring objects based on facts of the system.

Grafana Loki demo
Directly from Grafana Labs represented by Ganesh Vernekar the audience got some news about Loki which is “Like Prometheus, but for Logs”. So Loki avoid huge indexes and allows for better scaling by not indexing log lines but grouping them to streams. Using a similar format to Prometheus it allows to get metrics and logs for a system without a context switch. Running Loki to get this seems quite simple and flexible with only one binary which can scale out easily and also allows for a microservice infrastructure.

First code improvement
“Fast Logs Ingestion” by Nicolas Fränkel showed common coding mistakes and how to avoid them to get better logging in your application. He also covered topics like metadata and searching logs which also should also influence decisions and code. Structured log data even not written to file are also a option to consider like config reload during runtime to enable the user to switch loglevel without downtime. In the end he hopes people take away from his talk that everyone from dev and ops to architects should keep in mind which trade-off between speed and reliability has to be done and why.

Like every year the “Current State of Icinga” by Bernd was held in front of full house. He started with a short introduction to Icinga including the workflow which results in “Icinga makes you happy”. Icinga Workflow Afterwards to start with technical things he looked into the big changes Icinga 2.11 brought with a new network stack, high availability for more features and a new process handling not only helping with containers. Icinga 2.7 brought more translations, markdown support, jQuery 3, modernized styling for forms and lists, color blind theme and improvements for module developers. The vSphere module provides now an Import Source for Director, no code depenency on the Director and some UI improvements. The latest version of Director has also more translation, support for scheduled downtimes and sync previews. The BP Modelling (formerly Business Process Modul) has now drag & drop, export and import and breadcrumbs to make the UI more usable. As the first new feature he introduced the Windows monitoring Christian gave a detailed talk earlier today. Icinga for AWS was an improvement to the one only providing a simple import source for Director which adds support for multiple sources, some property modifiers and sync previews. Icinga Module for Jira includes an Issue overview, Jira notification via Director integration and custom workflows you can create from Icinga Web 2. Icinga DB as replacement for IDO is decoupling status and historic data using Redis and in a demo the new monitoring module based on it was also shown including all the visual improvements. Pull requests are already merged and will be part of the next releases and new things are available separately. The next update you can get on IcingaConf in Amsterdam on May 12 – 14, 2020.

As last topic a open discussion about Code of Conducts suggested and moderated by Stefan Lange took place.

Pictures are taken from several twitter users tagging them with OSMC. Thanks for providing them, expect some better ones from my colleagues from the events and marketing teams. I hope you enjoyed my report for day 1 while I am heading over to the social event at the Loftwerk and try to have at least a short talk to everyone. Day 2 will be covered tomorrow evening.

Dirk Götz
Dirk Götz
Principal Consultant

Dirk ist Red Hat Spezialist und arbeitet bei NETWAYS im Bereich Consulting für Icinga, Puppet, Ansible, Foreman und andere Systems-Management-Lösungen. Früher war er bei einem Träger der gesetzlichen Rentenversicherung als Senior Administrator beschäftigt und auch für die Ausbildung der Azubis verantwortlich wie nun bei NETWAYS.