Here you can find all videos and slides of the OSDC 2015:
TOOLS & METHODS
Nigel Kersten | In Defense of Data Centers
Modern operations practices and Devops-style methodologies don’t have to require the adoption of public cloud services. For some environments, making a modern data center the core of your operations is the most flexible and appropriate choice if you ensure the implementation of some of the same constraints as the public cloud.
In this talk we’ll sketch out how automated modern operations practices can transform your operational efficiency in data centers without necessarily requiring the use of public cloud, private cloud, or even virtualization.
Luca Gibelli | Skylable: Storing Massive Amounts of Data, with Minimal Effort
Skylable provides an object storage solution which you can run on your own hardware. It provides data replication, deduplication, client-side encryption, and built-in data integrity checks. Skylable scales easily to hundreds of TBs and it can be deployed in literally 2 minutes by anyone with a minimal experience of Linux administration.
The installation and setup process is straightforward and requires very few dependencies.
Skylable is fully compatible with Amazon S3 APIs. It also offers a more advanced RESTful protocol called SX, which solves many limitations of the S3 protocol, most importantly the ability to upload/download files in parallel from multiple hosts and true support for resuming transfers.
The software is released under the GPL licence, the client library (libsx) is LGPL.
This talk is about the ease of use of Skylable, a comparison of S3 vs SX and the real-world advantage of doing deduplication and multiplexed transfers.
Mitchell Hashimoto | Automating the Modern Datacenter, Development to Production
Physical, virtual, containers. Public cloud, private cloud, hybrid cloud. IaaS, PaaS, SaaS. These are the choices that we're faced with when architecting a datacenter of today. And the choice is not one or the other; it is often a combination of many of these. How do we remain in control of our datacenters? How do we deploy and configure software, manage change across disparate systems, and enforce policy/security? How do we do this in a way that operations engineers and developers alike can rejoice in the processes and workflow?
In this talk, I will discuss the problems faced by the modern datacenter, and how a set of open source tools including Vagrant, Packer, Consul, and Terraform can be used to tame the rising complexity curve and provide solutions for these problems.
Roland Kammerer | DRBD9: Managing High-Available Storage in Many-Node Setups
Recent publications show an ever increasing demand in the area of cloud computing where high-available storage is one important corner stone.
DRBD (Distributed Replicated Block Device) is a building block for high availability clusters since years. Currently, DRBD is basically limited to two node cluster setups. In this talk we will provide an overview about recent developments in DRBD9 that allow us to make DRBD ready for upcoming challenges like many-node cluster setups and highly automated cloud deployments.
For the upcoming release we added an abstraction layer which is handled by “drbdmanage”. It is a tool that takes over management of logical volumes (LVM) and management of configuration files for DRBD. Features of drbdmanage include creating, resizing, and removing of replicated volumes. Additionally, drbdmanage handles taking snapshots and creating volumes in consistency groups. In order to support cloud deployments, a cinder (OpenStack) driver is in development.
All the mentioned components are currently under active development and this talk will provide an overview about these tools as well as about our vision for DRBD in general.
With the upcoming DRBD9 release we will be able to support a higher number of nodes (up to 30) per replication group, ease the deployment of DRBD setups with drbdmanage (up to 1000 nodes planed) and provide the foundations for cloud integration.
Kelsey Hightower | Building Distributed Systems with CoreOS
Marco Ceppi | Managing Workloads at Scale in the Cloud or on Bare Metal with Juju
Whether you're deploying Hadoop and Big Data workloads, OpenStack at High Availability, or your application. Juju makes these tasks and more easy to manage, scale, and repeat - doesn't matter if you're on a public cloud, private cloud, or building your own cloud on metal.
In this talk we'll deploy OpenStack and Hortonworks Hadoop on bare metal and in a cloud, discuss what/how Juju works, and explore other possible solution/workloads which Juju can manage.
Juju is Canonical's next generation cloud orchestration tool provided as free and open source. Written in Golang, it provides fast orchestration for any workload. Juju makes use of an orchestration node in the environment which marshalls events that occur in the environment and dispatches them to the extremely lightweight agents installed on each machine.
The orchestration node also acts as the entry point for communication to the environment via a websocket API. The CLI and the GUI (both open source software) use it for communication to an environment.
James Fryman | DevOps Next Steps: Event Driven Operation
The most critical component of building out successful DevOps culture is creating a successful merger of sustainable technology and process. ChatOps practices help teams accelerate delivery by bringing in technology operations inline with conversations already happening today. This in turn helps turns operations into simple events than can be chained together to make teams even more powerful and flexible than ever before! This talk discusses the practical implementations of ChatOps and how it can be used to grow teams of all types and sizes.
Jan-Piet Mens | MQTT for your Data Center (and for the IoT)
MQTT is a PUB/SUB protocol for the Internet of Things, but it's also valuable for systems administration and for your data center. We'll take a close look at MQTT and its infrastructure, and we'll show you how to use a microcontroller to monitor your server-room's temperature with it, publishing and monitoring it via MQTT and Icinga/Nagios. Furthermore, we'll also show you some real-world integrations of MQTT and Unix/Linux system utilities.
Bernd Mathiske | Why the Datacenter Needs an Operating System
Developers are moving away from their host-based patterns and adopting a new mindset around the idea that the datacenter is the computer. It?s quickly becoming a mainstream model that you can view a warehouse full of servers as a single computer (with terabytes of memory and tens of thousands of cores). There is a key missing piece, which is an operating system for the datacenter (DCOS), which would provide the same OS functionality and core OS abstractions across thousands of machines that an OS provides on a single machine today. In this session, we will discuss:
How the abstraction of an OS has evolved over time and can cleanly scale to spand thousands of machines in a datacenter.
How key open source technologies like the Apache Mesos distributed systems kernel provide the key underpinnings for a DCOS.
How developers can layer core system services on top of a distributed systems kernel, including an init system (Marathon), cron (Chronos), service discovery (DNS), and storage (HDFS)
What would the interface to the DCOS look like? How would you use it?
How you would install and operate datacenter services, including Apache Spark, Apache Cassandra, Apache Kafka, Apache Hadoop, Apache YARN, Apache HDFS, and Google's Kubernetes.
How will developers build datacenter-scale apps, programmed against the datacenter OS like it?s a single machine?
Bernd Ahlers | What is your Configuration Management System doing?
If you are managing computing infrastructure these days you probably use some kind of configuration management system like Chef, Puppet, Ansible or others. But do you know what those systems do all day long? What changes are applied over the day? Did something change on a machine that caused an incident? Most configuration management systems have pluggable event reporting mechanisms. This talk will show you how to use Graylog2, Icinga2 and a custom Chef handler to collect data from Chef runs and Icinga2 monitoring events of all your machines in a central place. This allows you to see what changes are applied on your infrastructure, alert on certain changes or even correlate monitoring alerts with changes on a machine.
We will provide virtual machine images after the talk to make it easy to get started with the presented examples.
David Norton | InfluxDB - Scalable Metrics Made Easy
There are many applications that need an easy way to store and analyze periodic measurements, whether it's system stats from multiple data centers or a Saturday afternoon project. Learn how InfluxDB can be used to store and analyze metrics as your system scales.
This session will be an introduction to InfluxDB, an open-source, distributed, time series database. It's written in Golang and compiles to a single executable, making deployment simple. We'll see how a fresh install of a single instance can be up and running in minutes. We'll look at its SQL-like query syntax, designed to reduce the learning curve coming from other SQL databases. And, we'll see how to add nodes to create a cluster as a system grows.
Tudor Golubenco | Application Performance Management with Packetbeat, Elasticsearch and Kibana
Typical Application Performance Management (APM) solutions like New Relic, AppDynamics or Compuware are closed source because they require significant up-front development efforts. Due to the recent advances in open source storage and analytics technologies, it is now feasible to create a generic APM that is more applicable, more flexible and easier to integrate than the commercial alternatives.
The talk will present the requirements for a good open source APM solution and the proposed Packetbeat architecture to meet those requirements. It will demo our automation scripts for installing the complete Packetbeat system and it will show how to use it to monitor the performance of an example Django application. It will also introduce Bonito, a new web interface for the Packetbeat data which is complementary to Kibana. Finally, the talk will explain the differences in overhead and features when compared to the commercially available tools.
Stephen Benjamin | Foreman in Your Data Center
Timo Derstappen | Microservice Orchestration
There’s no doubt, everyone’s talking about container technologies these days. Especially Docker has established itself as a de facto standard here. These technologies are especially suited to develop, run, and deploy microservices. So there are a few things coming together. This talk will give an overview why things like immutable infrastructure, continuous delivery and small autonomous teams make a lot of sense in today’s web scale architectures. Further, we discuss how these technologies can be used to manage an infrastructure with a high amount of containers.
Martin Loschwitz / Kristian Köhntopp | 45 Minutes of OpenStack Hate
OpenStack has been dominating the news on Open Source cloud computing for more than two years now - and there is no end in sight for the hype. If you have been looking into cloud computing, you will most likely have considered OpenStack as a possible solution. You will also have heard success stories of large organizations such as Rackspace or CERN. And of course people told you about all the glittering parties held during the semi-annual OpenStack summits.
What you probably haven't heard that often are stories about all the occasions where OpenStack will blow up right in your face. At SysEleven, we've been working on an OpenStack platform for more than a year now - and we would like to share our experiences with you in this presentation. We'll explain why we have decided to go with OpenStack in the first place, what problems we have ran into and how we solved them. We'll demonstrate what our platform looks like at the moment and what challenges we are currently working on. At the end, you will have a better understanding of what OpenStack means for ISPs and what kind of trouble you are signing up for when becoming an OpenStacker.
Ingo Friepoertner | Polyglot Persistence & Multi-Model NoSQL Databases
Kris Buytaert | From ConfigManagementSucks to ConfigManagementLove
Matthias Klein | How to use Open Source Software to have near Production Development and Testing Environments
- apt-mirror - why is it a good idea to have multiple own repositories
- puppet/hiera - the tool that glues everything together
- vagrant - create production like machines for developers in nearly no time
- jenkins - test and build your software
- dpkg-deb - why you should deploy using packages
John Spray | The Ceph Storage System
Georg Schönberger | Linux Performance Profiling and Monitoring
Nowadays system administrators have great choices when it comes down to performance profiling and monitoring. The challenge is to pick the ppropriate tool and interpret their results correctly.
This talk is a chance to take a tour through various performance profiling and benchmarking tools, focusing on their benefit for every sysadmin. The topics will range from simple application profiling over sysstat utilities to low-level tracing methods. Besides traditional Linux methods a short glance at MySQL and Linux containers will be taken, too, as they are widely spread technologies.
At the end the goal is to gather reference points to look at, if you are faced with performance problems. Take the chance to close your knowledge gaps and learn how to get the most out of your system.
Pere Urbon | Scaling Logstash: A Collection of War Stories
In this talk, we will cover several strategies for successfully scaling Logstash. Through the lens of several real-life war stories, you willl learn how to make Logstash sing alongside RabbitMQ, Redis, ZeroMQ, Kafka and much more. If you are ready to grow at scale and make your infrastructure more resilient, this talk is for you.
Dr. Udo Seidel | Developing Applications for the New Cloud Operating Systems
Along with many other things the cloud has triggered the creation of new opensource operating systems. Some of them are still a kind of a Linux. Others have broken with many traditions in order to be the best O/S for the cloud. Even the more Linux-like ones require a change of thinking regarding setup and management. This has a knock-on effect on how to develop applications which should run on these new operating systems. Using CoreOS, OSv and MirageOS this talk will describe the design and architecture of the newcomers in the cloud O/S family. This is followed by the changes needed on the application development side. The selected examples represent the broad spectrum from 'almost no code changes needed' to 'rewrite from scratch'.
Benoit Peccatte | Sharing IT Automation Benefits in a Team with Rudder
Implementing a tool to automate IT infrastructure management has many undeniable benefits and is quickly becoming an industry standard. But using a tool like this usually has a pretty significant impact: new processes, new language(s) to learn, new way of doing pretty much everything on your infrastructure. Doing so in coordination with different profiles having a different view on the same infrastructure can be challenging. Moreover, there is a big risk that a "devops hero" holding the project becomes a bottleneck of the new process.
Some tools need a minimal skillset that must be acquired before using them. That is not an option for everyone in the team when they don't have enough time and willingness dedicated to learning a new language. Rudder is an easy to use, web-driven, role-based solution for IT Infrastructure Automation & Compliance. As such it lets different user profiles, from technical experts to manager and junior sysadmin concentrate on their part of the work and gives feedback on the infrastructure status at different levels.
This talk will show the advantages of Rudder when working in a team, how this product helps working together on infrastructure automation. You will see a real world scenario with a project carried out by different profiles such as an IT automation expert, a manager or a security officer, collaborating to install and use Rudder on an existing platform.
Bernd Erk | Why favour Icinga over Nagios
Most sys admins have a love-hate relationship with Nagios based monitoring solutions. Backed by a sizable community, users have learned to live with it’s shortcomings in scaling, configuration, and modern integration options.
Taking advantage of the tremendous number of supported hard- and software, Icinga leaves all legacy limitations behind. It delivers an easily scalable solution, with clustering, load balancing, automated replication, and even business process monitoring out-of-the-box. Based on a new configuration format with advanced language features - like conditional processing and complex type support - monitoring agile environments works like a breeze. Existing modules for Puppet, Chef and Ansible ramp up the rollout time and ensure a continuous and up to date monitoring environment.
The talk will demonstrate how popular tools such as Graphite, Logstash, or Graylog integrate better and easier than ever before. In addition to that we’ll introduce the new Icinga Web 2 interface and give a brief introduction into the technical architecture.