NETWAYS Blog

I’ve seen and learned plenty of programming languages either during my studies or in work or spare time related projects – be it C/C++/C#, VHDL, Java or PHP/Perl/Python/Ruby even (I’ve removed some in my XING profile to reduce recruiter spam level ;)). Choosing the „right“ language is always hard but most of the time the requirements of existing software or newly designed projects allow you to skip that part and already have one in mind.
When joining NETWAYS in late 2012, I took over the LConf backend project being entirely written in Perl similar to other plugins and scripts floating around as open source software. Icinga 1.x Core is partially using Perl as well (although embedded Perl in C is a horrible mess). Most recently our development team decided to go for Python as the primary programming language.
So, my Python foo wasn’t that good after some years not really using it. Learning from my colleagues and looking into additional ressources helped a lot. I also found „Head first Python“ from O’Reilly in my bookshelf as a gift from the Nagios/Icinga cookbook review which also provides an extensive introduction into Python – when you already know a scripting language like Perl.
We’ve been working on a customer’s project developing a plugin framework for executing checks via a defined transport (e.g. ssh) and parsing cli output. Find some collected hints and tricks I’ve learned below.

Remove colors from shell output

The EMC „isi“ cli command puts colors into the shell output (e.g. for a critical state turning the background red). While one could disable color support on the shell, this was not possible in that environment. The fix by Gunnar was easy: Parsing text output into a list called „lines“ and clean the line string from shell color codes:

lines = text.split('\n')
lines = [re.sub('\x1b\[[0-9;]*m', '', line).strip() for line in lines]

Convert datetime to unix timestamp

Consider having a datetime string whose output format is not fixed, which means some strftime print magic does not work for proper parsing. Luckily there’s a Python module called dateutil providing the required functionality.

import time
import datetime
import dateutil.parser
def datetime2unixts(time_str):
    dt = dateutil.parser.parse(time_str)
    return time.mktime(dt.timetuple())

$ python
Python 2.7.8 (default, Apr 15 2015, 09:26:43)
[GCC 4.9.2 20150212 (Red Hat 4.9.2-6)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import time, datetime, dateutil.parser
>>> def datetime2unixts(time_str):
...     dt = dateutil.parser.parse(time_str)
...     return time.mktime(dt.timetuple())
...
>>> time_str = "Wed May 13 16:15:34 CEST 2015"
>>> datetime2unixts(time_str)
1431526534.0

Initialize nested dictionary

Similar to what I am used to do with Perl and hashes I was stumbling over this with Python. In Perl one does not care about initializers, but just keeps populating all items like so:

my $hash = {}
$hash{'key1'}{'key2'} = "val1";

In Python this does not work out-of-the-box. There are some modules available such as „collectors“, but I prefer a somewhat native implementation making it work on older distributions. I came across this solution which I consider pure awesome 🙂

class AutoVivification(dict):
    """Implementation of perl's autovivification feature."""
    def __getitem__(self, item):
        try:
            return dict.__getitem__(self, item)
        except KeyError:
            value = self[item] = type(self)()
            return value
hash = AutoVivification()
hash["key1"]["key2"] = "val1"

Note: It certainly does create non-existing keys if you are checking for their existance. My implementation does not care about key checks, but requires known-to-exist values from a flat dictionary with „id_label“ as key stashed into a nested dictionary where id and label are separated/nested for further operations.

        jobs = AutoVivification()  # use a pre-initialized nested dict
        for k, v in sorted(perfdata.iteritems()):
            (job_id, label) = k.split("_", 1)
            if label == "ended_ts":
                jobs[job_id][label] = float(v)
            if label == "started_ts":
                jobs[job_id][label] = float(v)

Fix the indent

If your editor doesn’t do that automatically (e.g. converting tabs into 4 spaces) it might still work, but could also cause weird behaviour in Python depending on the indent only (no brackets as I am used to with Perl). Fixing this is fairly easy by using autopep8 – I’m using Fedora 22:

dnf install python-autopep8
autopep8 check_plugin -i

Note: ‚-i‘ replaces the given file with all the changes. Make sure to commit your other changes to git before.

Conclusion

I never thought that I would dig deep in Python again that easy. I used to hack that in the past with Win2k usb driver test frameworks, and also checkmk plugins, but gaining back experience worked out pretty well.
Icinga 2’s code, value types and configuration look a lot like Python as well – take dictionaries for custom attributes using apply for loops, eh? (Hint: Testdrive this config snippet inside the Icinga 2 Docker container).

object Host "dns" {
  import "generic-host"
  address = "127.0.0.1"
  address6 = "::1"
  vars.dns_checks["dns icinga.org"] = {
    dns_lookup = "icinga.org"
    dns_server = "ns1.netways.de"
    dns_expected_answers = "185.11.254.83"
  }
  vars.dns_checks["dns netways.org"] = {
    dns_lookup = "netways.org"
    dns_server = "ns1.netways.de"
    dns_expected_answers = "185.11.252.37"
  }
}
apply Service for (dns_check => config in host.vars.dns_checks) {
  check_interval = 1m
  retry_interval = 30s
  check_command = "dns"
  vars += config
}

Icinga 2, plugins, your project – here I come 🙂
PS: See you at the OSMC hackathon.