DevOpsDays Berlin: Call for Papers open. Get involved!

The organizers of DevOpsDays Berlin once again invite IT professionals with DevOps experience to propose a talk and show their interest in contributing. DevOpsDays is a technical conference, aimed at developers, sysadmins and anyone else involved in technology, whether expert or beginner. The idea behind it: Understanding each others way of working, improving code and cooperation. It was the series of DevOpsDays conferences that made the term DevOps as popular as it is today.

The Call for Papers for DevOpsDays Berlin runs until July 05, 2020. Papers can be submitted at cfp.devopsdays.berlin/2020/.

If you have a hiring success story, can talk about diversity in teams or leadership, have a great tale to tell about spreading DevOps to the rest of your company or organization: Don’t hesitate to propose.

There are three different formats

  • 30-minute talk: Presented during the conference, usually in the mornings. Anyone can submit a presentation. The DevopsDays Berlin program council will choose the best presentations among these submissions, which will appear in the agenda of the conference.
  • Ignite talk: Presented during the Ignite sessions (scheduling varies). The ignite talks are a valuable portion of knowledge, lasting for 5 minutes. 20 slides change automatically every 15 seconds giving a condensed overview on a topic.
  • Hands-On session: Presentations of technical nature lasting for 45 minutes. Speakers should demonstrate a part of their work and give people the chance to ask questions. No product demos allowed.

There will also be Open Space sessions during the conference. The Open Space format is a very interesting and simple one. Short time before the Open Space starts, all attendees can suggest topics and vote on what topic is the most wanted. After this, participants create discussion groups. It is not necessary to propose ahead of time. The topics are suggested in person at the conference.

Get involved!

DevOpsDays events are unique in that they are entirely volunteer-supported, and not hosted by an industry vendor or for profit. If you are in any way interested in supporting that, be it by speaking or by volunteering at the event: Reach out to the organizers. Get involved!

For more information visit https://devopsdays.org/events/2020-berlin/welcome/

Julia Hornung
Julia Hornung
Marketing Manager

Julia ist seit Juni 2018 Mitglied der NETWAYS Family. Vor ihrer Zeit in unserem Marketing Team hat sie als Journalistin und in der freien Theaterszene gearbeitet. Ihre Leidenschaft gilt gutem Storytelling, klarer Sprache und ausgefeilten Texten. Privat widmet sie sich dem Klettern und ihrer Ausbildung zur Yogalehrerin.

tmux – terminal multiplexer

In diesem Artikel möchte ich euch kurz das Tool tmux vorstellen, dass zu meinen Lieblingstools unter Linux gehört. In einem vorherigen Artikel hat Johannes schon einmal die Basisconfig von tmux gezeigt.

Da tmux eine Server Client Struktur hat, bevorzuge ich es in der Regel die Finger von der tmux config zu lassen. Da man die sonst auf jedem Server pflegen muss, mit dem man arbeiten möchte. Leicht hat es an der Stelle derjenige, der Puppet oder Ansible nutzt und seine eigenen Server pflegt. Ich als Consultant habe meistens Kundenserver, auf denen ich nichts verändern möchte was nicht nötig ist.

tmux hat für mich 2 Einsatzzwecke:

  • Bei Präsentationen Bildschirm- und Beamerterminal synchronisieren.
  • Bei Servern meine Sessions schützen, so dass ich nach Verbindungsabbrüchen weiterarbeiten kann.

Besonders der zweite Zweck ist sehr praktisch, da ich auch Bahnfahrer bin.

CLI

Jetzt zur eigentlichen Benutzung: Gestartet wird tmux über den jeweiligen User. Wenn man ein Terminal vor sich hat sind folgende Basisbefehle praktisch:

  • # tmux -> startet eine neue unbenannte Session
  • # tmux new -s “MeineSession” -> startet eine benannte Session
  • # tmux list-sessions -> listet Sessions
  • # tmux -a -> verbindet zu einer unbenanntenten Session
  • # tmux -a -t “MeineSession”

Inside tmux

Wenn wir jetzt in einer tmux Session sind kann das folgendermaßen aussehen:

tmux ScreenshotTmux unterscheidet zwischen “window” und “pane”. Wir sehen in diesem Beispiel 4 panes in einem window.
Um innerhalb von tmux zu navigieren braucht gibt es diverse Keybindings die immer mit Strg+b eingeleitet werden. Hinweis: Bei screen, einem ähnlichen aber älteren tool als tmux, war es Strg+a. Wem das lieber ist, der kann bei Johannes nachschlagen, wie man das umstellt.

Im Window hat man folgende Befehle:

  • Strg-b, % Horizontaler Split
  • Strg-b, ” Vertikaler Split
  • Strg-b, x Schließt den aktuellen Pane
  • Strg-b, <Pfeil-Tasten> Zwischen Panes hin und her springen
  • Strg-b, d detach, von tmux abmelden ohne es zu beenden

Ein Window ist vergleichbar mit einem Tab in deinem Browser. Zwischen einzelnen Windows navigiert man mit:

  • Strg-b, c Neues Window
  • Strg-b, w Windows auflisten
  • Strg-b, [0…9] zum Window Nr. 0-9 wechseln

Kopieren

Ein praktisches Feature ist das kopieren. Das funktioniert ähnlich wie in Vim aber, da es Terminals sind, auch zwischen verschiedenen Servern.

  1. Als Erstes braucht man mindestens 2 Panes.
  2. Strg-b, [ Öffnet den Scroll-Modus
  3. <Space> startet den Markier-Modus unterm Cursor
  4. <CR/Enter> kopiert
  5. Nach dem Wechsel ins nächste Pane . . .
  6. Strg-b, ] fügt ein

Am besten probiert man das alles selber mal aus. Es gibt auch eine Menge Cheat-Sheets zu dem Thema, die vor allem am Anfang hilfreich sein können.

Christoph Niemann
Christoph Niemann
Senior Consultant

Christoph hat bei uns im Bereich Managed Service begonnen und sich dort intensiv mit dem internen Monitoring auseinandergesetzt. Seit 2011 ist er nun im Consulting aktiv und unterstützt unsere Kunden vor Ort bei größeren Monitoring-Projekten und PERL-Developer-Hells.

Ansible – should I use omit filter?

When we talk about Ansible, we more and more talk about AWX or Tower. This Tool comes in handy when you work with Ansible in a environment shared with colleagues or multiple teams.
In AWX we can reuse the playbooks we developed and share them with our colleagues on a GUI Platform.

Often we need a bit of understanding how a playbook is designed or if a variable need to be defined for the particular play. This can be much more tricky when sharing templates to people unaware of your work.

This is where the omit filter can be used. The easiest way to explain this, if the variable has no content or isn’t defined, omit the parameter.

The following example is an extract from the documentation:


- name: touch files with an optional mode
  file:
    dest: "{{ item.path }}"
    state: touch
    mode: "{{ item.mode | default(omit) }}"
  loop:
    - path: /tmp/foo
    - path: /tmp/bar
    - path: /tmp/baz
      mode: "0444"

In AWX we can create surveys, those are great to ask a few questions and provide a guide on how to use the underlying play. But often we need to choose between two variables whether one or another action should happen. Defined by the variable in use. If we leave one of both empty, Ansible will see those empty as defined but “None” (Python null) as content.

With the omit filter we can remove the parameter from the play, so if the parameter is empty it won’t be used.

The following code is the usage of icinga2_downtimes module which can create downtimes for hosts or hostgroups but the parameters cannot be used at the same time. In this case I can show the variable for hostnames and hostgroups in the webinterface. The user will use one variable and the other variable will be removed and therefore no errors occur.


- name: schedule downtimes
  icinga2_downtimes:
    host: https://icingaweb2.localdomain
    username: icinga_downtime
    password: "{{ icinga_downtime_password }}"
    hostnames: "{{ icinga2_downtimes_hostnames | default(omit) }}"
    hostgroups: "{{ icinga2_downtimes_hostgroups | default(omit) }}"
    all_services: "{{ icinga2_downtimes_allservices | default(False) }}"

The variables shown in the AWX GUI on the template.

This filter can be used in various other locations to provide optional parameters to your users.

If you want to learn more about Ansible, checkout our Ansible Trainings or read more on our blogpost.

Thilo Wening
Thilo Wening
Consultant

Thilo hat bei NETWAYS mit der Ausbildung zum Fachinformatiker, Schwerpunkt Systemadministration begonnen und unterstützt nun nach erfolgreich bestandener Prüfung tatkräftig die Kollegen im Consulting. In seiner Freizeit ist er athletisch in der Senkrechten unterwegs und stählt seine Muskeln beim Bouldern. Als richtiger Profi macht er das natürlich am liebsten in der Natur und geht nur noch in Ausnahmefällen in die Kletterhalle.
From Monitoring to Observability: Distributed Tracing with Jaeger

From Monitoring to Observability: Distributed Tracing with Jaeger

Modern cloud environments and microservice architectures need a changed mindset when it comes to monitoring. Classic host/service object relations are not always applicable, containers run in Kubernetes are short lived, and application performance within distributed networks is hard to monitor. Especially with applications running millions of operations, where to start the root cause analysis for slow client responses in your web shop? Is it just slow connections to MySQL, or does the application’s debug log slow down the entire fleet?

This is where observability with tracing comes into play. In the cloud native space, OpenTracing evolved as vendor neutral standardized API including client instrumentation. Famous tools are Zipkin and Jaeger which was contributed from Uber to the CNCF.

Let’s have a look into Jaeger today.

 

Getting Started

The easiest way to try Jaeger is with using the Docker container explained in the documentation.

docker run -d --name jaeger \
  -e COLLECTOR_ZIPKIN_HTTP_PORT=9411 \
  -p 5775:5775/udp \
  -p 6831:6831/udp \
  -p 6832:6832/udp \
  -p 5778:5778 \
  -p 16686:16686 \
  -p 14268:14268 \
  -p 9411:9411 \
  jaegertracing/all-in-one:1.16

Navigate to http://localhost:16686 to get greeted by Jaeger.

 

Try it

A sample application is available as container. I’m using a modified port mapping with 8081-8084 here since port 8080 is already assigned.

docker run --rm -it \
  --link jaeger \
  -p8081-8084:8080-8083 \
  -e JAEGER_AGENT_HOST="jaeger" \
  jaegertracing/example-hotrod:1.16 \
  all

Navigate to http://localhost:8081 and click the buttons to order some cars.

Within Jaeger itself, start analyzing the traces and for example learn that Redis runs into timeouts quite often delaying the HTTP responses to the clients.

 

Traces for applications

Jaeger provides officially supported client libraries for Go, Java, Python, NodeJS, C/C++, C#/.NET, others are in the making. Let’s see if we can add it into Icinga 2 and do some tracing here.

First off, clone the repository, build and install the client libraries. You’ll need CMake, a C++ compiler, etc. – basically everything which is required for Icinga 2 too and documented here. In this example, I’m compiling on my Macbook. There are additional libraries and headers required. Hint: Boost 1.72 has a bug which needs a patch.

brew install yaml-cpp thrift 
git clone https://github.com/opentracing/opentracing-cpp && cd opentracing-cpp
# 1.6.0 doesn't work atm
git checkout v1.5.1
mkdir -p build && cd build
cmake ..
make && make install
cd ..

Then clone, cmake, make, install.

git clone https://github.com/jaegertracing/jaeger-client-cpp && cd jaeger-client-cpp
git checkout v0.5.0
# regenerate thrift headers for 0.11.0
find idl/thrift/ -type f -name \*.thrift -exec thrift -gen cpp -out src/jaegertracing/thrift-gen {} \;
mkdir -p build && cd build
cmake ..
make && make install
cd ..

In order to add Jaeger into Icinga 2, there are the following steps necessary:

  • Add CMake functions to lookup yaml-cpp, opentracing, jaeger headers and libraries
  • Optionally enable Jaeger tracing code, link the icinga2 binary against it
  • Add a new tracer into the Config Compiler CLI command to measure the timing points

The majority of development time today was to resolve compile and linking issues, adding spans and traces is really simple. You can follow my progress in this Icinga PR – developers, get started wtih the client libraries and help your colleagues with enhanced observability!

 

Conclusion

Tracing application performance, cluster messages, end2end tests and any sort of event span provides valuable insights for both, devs and ops. With the new cloud native landscape evolving fast, we have additional possibilities to analyze our environments. Next to the now standardized tools for parsing and ingesting logs with Elastic Stack or Graylog, tracing has found its place in the observability stack.

Jaeger Tracing also is part of the GitLab observability suite which will be moved to the free core edition in 2020. Metrics, logging, alerts and tracing are key elements in modern cloud environments. Prometheus monitors everything from classic load checks to Kubernetes containers, Jaeger provides application performance insights and on top of that, Grafana combines the view on problems and trends. You can learn more about GitLab’s vision here.

Exploring these cool new features in GitLab are our mission in future trainings and workshops, watch this space in 2020!

GitLab-CI / YAML – Write less with Anchors, Extends and Hidden Keys

GitLab-CI / YAML – Write less with Anchors, Extends and Hidden Keys

Have you ever wanted to execute a GitLab-CI job for multiple operating systems and just copied every line of YAML multiple times?
Anchors, extends and hidden keys are coming to rescue!

Let’s say you have two jobs and the only difference between them being a single environment variable:

stages:
  - echo

echo-hello:
  stage: echo
  script:
    - echo $ECHO_TEXT
  variables: 
    ECHO_TEXT: "Hello world!"

echo-bye:
  stage: echo
  script:
    - echo $ECHO_TEXT
  variables: 
    ECHO_TEXT: "Bye bye!"

Anchors and extends

Writing the same job two times can already get quite messy and hard to maintain. The more jobs you add, the worse it gets.
But don’t worry, YAML has got you covered. Anchors and extends let you reuse parts of your config and extend on them.

In this example, we create the echo-hello job and extend on it in the echo-bye task:

stages:
  - echo

echo-hello: &echo #create an anchor named "echo"
  stage: echo
  script:
    - echo $ECHO_TEXT
  variables: 
    ECHO_TEXT: "Hello world!"

echo-bye:
  <<: *echo #use the anchor created above and extend it by using "<<"
  variables: 
    ECHO_TEXT: "Bye bye!"

Templating with hidden keys

One thing you can do to further improve on that is, by using a separate task just for templating using hidden keys.
Hidden keys can be defined in YAML using a . in front of a keys name. This prevents GitLab-CI from executing a job and allows us to use it as a template.

In our last example, we create an echo template job containing our stage and script. The echo job is then extended on in echo-hello and echo-bye:

stages:
 - echo

.echo: &echo #keys (jobs in this case) with a dot in front are hidden keys and won't be executed by GitLab
  stage: echo 
  script: 
    - echo $ECHO_TEXT  

echo-hello: 
  <<: *echo 
  variables:  
    ECHO_TEXT: "Hello world!"  

echo-bye: 
  <<: *echo  
  variables: 
    ECHO_TEXT: "Bye bye!"

Some real world examples can be found in our public Icinga 2 packaging repositories: https://git.icinga.com/packaging/rpm-icinga2

Noah Hilverling
Noah Hilverling
Developer

Nachdem Noah bei einer vierjährigen Exkursion nach Belgien seine Liebe zum Programmieren entdeckte, holte der gebürtige Euskirchener innerhalb kürzester Zeit gleich zwei Schulabschlüsse nach. Danach verließ Noah sogar den schönen Chiemsee, um sich ab September 2016 im Rahmen der Ausbildung zum Fachinformatiker für Anwendungsentwicklung bei NETWAYS voll und ganz dem Programmieren hinzugeben und viele unterschiedliche Erfahrungen zu sammeln. Wenn er mal nicht am Programmieren und Zocken ist, brettert er mit seinem Snowboard die Pisten runter,...

DevOpsDays Berlin: Last Tickets

On Nov 27 – 28 2019 the DevOpsDays Berlin are taking place!

Success factor open space

One of the success factors of previous DevOpsDays has been the open space format. It’s quick, it’s easy, it’s communicative. It fosters the sharing of ideas and insights in the good old fashion, in a non-technical way by just talking to each other. This doesn’t require any effort besides registering and showing up at the event.

Great international line-up

We have introduced the wonderful DevOpsDays line-up before, but let me drop some of these names again, because of the pleasant expectations they arise: Monica Sarbu (Elastic), Beth North, Daniel Löffelholz (ThoughtWorks), Leonardo di Donato (Sysdig), Jessica Anderson (Meltwater)… Or, put differently: Wrapped around the open spaces will be a great line-up of international speakers! Learn more about it here.

Register now, listen carefully and throw yourself into discussion!

Julia Hornung
Julia Hornung
Marketing Manager

Julia ist seit Juni 2018 Mitglied der NETWAYS Family. Vor ihrer Zeit in unserem Marketing Team hat sie als Journalistin und in der freien Theaterszene gearbeitet. Ihre Leidenschaft gilt gutem Storytelling, klarer Sprache und ausgefeilten Texten. Privat widmet sie sich dem Klettern und ihrer Ausbildung zur Yogalehrerin.