Here you can find all videos and slides of the OSDC 2013:
CLOUD & BIG DATA
DEVOPS & METHODS
Constantino Vázquez | The OpenNebula Cloud Platform for Datacenter Virtualization (ENG)
The presentation will describe the OpenNebula project for data center virtualization, with a focus on how to deploy OpenNebula-based private clouds and its unique features to tune and adapt it to any technology in the cloud and virtualization ecosystem.
The target audience are developers and IT administrators interested in deploying a private cloud solution, or in the integration of OpenNebula with other projects. The talk will be useful for both people with experience or without prior knowledge of OpenNebula, as it will start by introducing the project and its main features, along with a quick demonstration. Although this is not a hands-on tutorial, by the end of the presentation attendees will have a comprehensive idea of the integration and customization capabilities of OpenNebula in different areas, like user authentication, virtualization, storage, networking, etc.
Erkan Yanar | lxc@libvirt (DE)
This presentation shows how LXC can be managed by libvirt. This allows for example to manage LXContainer via virt-manager or to link OpenStack.
Nicholas Mailer | Deconstructing the Cloud (ENG)
Over the last few years, many people have been beguiled by the term "cloud computing". My talk will discuss how this term suffers from a dangerous ambiguity, which can subvert the very freedoms and flexibility that Open Source and Free Software provides. I provide examples, and discuss how Free Software and Open Source principles can be extended across a datacentre full of private "clouds" to avoid the problems that will otherwise become increasingly serious in our industry.
My talk deals with the philosophy, semantics, technology and politics of these issues, and should at least provide some food for thought and sparks for further discussion!
René Koch | Enterprise open source virtualization with oVirt and RHEV (DE)
Nowadays,virtualization is the foundation of many IT environments in medium and large enterprises. The KVM based Red Hat Enterprise virtualization and its upstream project oVirt, is a high performance and at the same time cost-saving open source solution combined with enterprise management features.
This presentation will introduce these virtualization solutions. With use-cases and field reports from the finance and industrial environment regarding KVM based desktop and server virtualization, we will outline the technical opportunities, applications scenarios and potential challenges.
Justin Clift | Tutorial and demonstration of failover from EC2 to OpenStack using Aeolus (ENG)
Martin Gerhard Loschwitz | Scale-Out made easy: Petabyte storage with Ceph (ENG)
Ceph is what you are looking for if you need seamlessly scalable storage: While traditional storage solutions like SANs or SAN drop-in replacements are scale-up only, Ceph allows you to turn every off-the-shelf computer into a member of a distributed, centrally managed storage network. With Ceph in place, you will never again have to worry about adding new disks to existing SANs or replacing existing disks with bigger ones -- just add new nodes to the cluster and see your storage grow. And as if that wasn't enough already, Ceph comes with inherent HA and replication capabilities, taking the burden of securing your data against outages off your shoulder.
This presentation gives an insight into the basic ideas behind the Ceph object storage solution. It also elaborates on the existing front-ends for Ceph (CephFS, RBD, radosgw) and explains how they work. Examples will demonstrate how to use Ceph in production. A live demo completes the talk.
Tugdual Grall | Introduction to NoSQL with Couchbase 2.0 (ENG)
You are getting ready to build a new webapp, but things are different this time. In the past, your number of users was certainly big, but this new app will be five to ten times the size and will need to support not just the browser, but mobile apps and be ready to deploy in both Europe and Asia in six months.
Thinking about how you'll hit these requirements, you've started to look into that new area of NoSQL. The options are many, but for the kind of interactive webapp you're looking to build, Couchbase Server looks pretty interesting. For instance, Draw Something scaled their mobile game to 50 million users in 50 days with Couchbase...
Couchbase Server, Open Source under an Apache 2.0 license, is built to be the scale-out data store behind the kinds of applications that want simplified development, high throughput and low latency and may need to scale at a moment's notice. Developers can store either JSON objects or binary data directly into a Couchbase Server cluster and are abstracted away from the details of managing how data is spread across the cluster. Scaling up a Couchbase cluster is simply a matter of tossing more servers (or VMs, EC2 instances, etc.) at an existing cluster and asking it to rebalance. Scaling down is equally easy. Changes to the cluster topology are abstracted away from the client application. The system is kept super-fast and highly available through the managed cache and the replication and reliable storage subsystem.
In this session, Tug Grall of Couchbase will provide a quick overview of Couchbase Server and then demonstrate growing a cluster behind a high-throughput, low latency workload. He'll then give a quick overview of how simple it is to develop an application for Couchbase, covering the simplified schema-less approach, definition of views and use of the Couchbase SDKs for all mainstream.
Dr. Udo Seidel | Distributed Storage with GlusterFS (DE)
So-called shared file systems are known for a long time for all IT administrators using Linux, Unix and Windows. The network based and also the cluster data systems are almost an old hat. Since a while distributed data systems become popular. The GlusterFS project is not new, but the community didn´t caught notice until the takeover of RedHat. Significant characteristics of this data system are the new approach to meta-data management and the new modular structure.
This presentation gives an insight into the approaches of data systems, and will explain the architecture and describe the first steps to set up a GlusterFS-cluster.
Olivier Renault | Introduction into Hadoop (ENG)
How do I deploy, manage and monitor Apache Hadoop within my Data Center? Olivier Renault will help you kick start your deployment by sharing the experience that the Hortonworks team have from deployment in the Yahoo Data Center and many other enterprise production deployments. During this talk, we will present Ambari the Open Source management solution by Apache which make deploying and monitoring Hadoop extremely easy.
Ohad Levy | Foreman - an interface for Puppet (EN)
Foreman is a complete lifecycle management tool for physical and virtual servers.
Through deep integration with configuration management and datacenter infrastructure (DHCP, DNS, PXE and Image based unattended installations), Foreman manages every stage of the lifecycle of your physical or virtual servers. Foreman provides comprehensive, auditable interaction facilities including a web frontend and robust, RESTful API.
Foreman can provision on bare-metal & public or private clouds all from one place with one simple process many different operating systems and is a complete configuration management solution including an ENC for Puppet, built-in support for parameterized classes and hierarchical parameter storage, Collect Puppet reports and facts. Monitor host configuration. Report status, distribution and trends.
Andy Hawkins | Introduction into Chef (EN)
This presentation will give an overview about what Chef is and how to access it. It will describe the typical use cases and architecture as well as Cookbooks, data bags and other concepts and will explain how to implement your CM solution. Finally it will show how to drive a successful Chef project.
Jan-Piet Mens | Ansible: configuration management doesn't have to be complicated (ENG)
Ansible is a simple configuration management and command execution framework for deployments for Unix/Linux systems using an existing SSH infrastructure. It's particularly easy to deploy because neither does it require an on managed nodes (a newish implementation of Python suffices) nor does it require a complex PKI. We show you how to quickly get started using Ansible for ad-hoc tasks, discuss some of its modules and introduce you to Ansible's playbooks and variables. We show you how to run Ansible as a normal user (non-root), how to configure inventory data, and give you sundry tips on using Ansible effectively. If you prefer a pull-based setup, we show you how to implement that as well.
Michael Prokop | Continuous Integration in data centers (DE)
Every developer is familiar with continuous integration; however, in data centers it is still largely disregarded. But many paradigms are transmittable and especially in times of “Infrastructure as a Code” with tools like Puppet, Chef, Vagrant & Co, also practicable. The advantages of automated documentation generation, standardized quality checks and controlled deployment, speak for themselves.
The follow up project of the Open Source Continuous Integration Server “Jenkins”, jenkins-debian-glue (http://jenkins-debian-glue.org/), allows continuous integration when applying Debian-/Ubuntu packages. With every commit in the version control system repository, you get automatically built Debian packages. Quality checks and upgrade checks allow the early recognition of risks, before resultant packages will be rolled out. By now, many renowned open source projects, like Grml (http://grml.org/), PostgreSQL (https://wiki.postgresql.org/wiki/Apt) und Icinga (http://icingabuild.dus.dg-i.net/), apply jenkins-debian-glue.
This presentation will show how you can realize continuous integration with Debian packages and which possibilities exist for quality checks.
Andrew Ford | Software Packaging with RPM Demystified (ENG)
There is a dearth of definitive information about packing with RPM. This talk will fill in those gaps, covering details of constructing a solid spec file, writing install scripts and triggers, aspects of packaging that need to be considered, and differences between versions of RPM corresponding to currently supported versions of Red Hat and SUSE enterprise distros, as well as Fedora and openSUSE.
Daniel Kirstenpfad | "We have Ignition" - Warum virtualisiertes Entwickeln und Testen keine Raketenwissenschaft ist (DE)
In fast-growing start-ups many problems occur which have not only an impact on the business but also on the development. Software and hardware are becoming more complex – in most cases, they change much faster than the documentation. Thus unexpected effects have often downtimes and time-consuming debugging as a consequence.
The virtualization of current production environments might make sense to get a grip on the documentation gaps and to ensure riskless further development for the production environment.
This presentation will demonstrate with the help of specific examples and field reports, how we increase the development speed, the quality and the safeness at Rakuten Deutschland by virtualizing the production environment.
Jan Gehring | Serverorchestrierung mit Rex (DE)
Rex is a tool to automate applications and configuration rollouts. The desired environment is described by a script (Perl) and can be implemented on different systems. Rex links up with the target system and executes the required work.
In this talk I first explain the basics of Rex. Afterwards, I will show how you develop for Rex Modules to ensure the reusability of codes. And finally I will demonstrate how provision further server in an automated manner with a performance monitoring (e.g. with Zabbix) and Rex, in case the load is getting too high for the current server.
Schlomo Schapiro | Configuration Management and Linux Packages (ENG)
What is the connection between configuration management (CM) and Linux packages? Is there a connection? Why do the Linux packages get in the way of CM all the time? Why should I care about this topic?
In the modern world everybody has some tooling for CM, be it one of chef/puppet/cfengine or something else. All CM tools basically serve the same purpose: Automate everything between a blank new machine and a running system that is ready for production. That includes installing some packages and changing configuration files.
Not really surprising, Linux packages actually serve a very similar purpose, though with a completely different objective. Packages also install other packages and also bring some configuration files.
That is the reason why we should talk about CM and packages: Two different tool sets doing the same job with different objectives. That is also the reason why CM was invented and why distro packages tend to get in the way of what you are trying to achieve with CM. Look at sysadvent.blogspot.de/2012/12/day-10-packages-doing-too-much.html for a good example of this conflict of interests.
When faced with the challenge of finding a new deployment and configuration management solution, we decided to try something different and radically new: Use Linux packages for configuration management!
In this talk I will explain the rationale behind that decision and the design choices that allowed us to do this. We believe that this is a way out of the CM-packages conflict because we actually use the same tooling for everything: OS deployment, software rollout and configuration rollout. In our world there cannot be a conflict between distro packages and configuration because we actively design our packages and configuration to work seamlessly together with the distro packages.
If the time permits I will be happy to share a live demo of how we work with configuration packages.
The result of our work is published under the GPL at yadt-project.org, our tooling to create config RPMs from configuration data snippets kept in SVN can be found in github.com/yadt/yadt-config-rpm-maker. Take this as an example, the important part is package-based configuration rollout. It doesn't matter how you actually create those packages, it is only important that they are dumb and contain no install-time scripts.
Lindsay Holmwood | Ript: making Linux firewall change management resilient (ENG)
Netfilter is an extremely powerful framework for manipulating packets, but does anyone enjoy using iptables? Tools for managing small rulesets have a steep learning curve, and most tools don't take availability into account when managing large rulesets.
Enter Ript, a clean and opinionated Domain Specific Language for describing firewall rules, and a tool that implements database migrations-like functionality for applying these rules with zero downtime. At Ript's core is an easy to use Ruby DSL for describing both simple and complex sets of iptables firewall rules, with helpers for all the common use cases: accepting, dropping, rejecting packets, as well as for performing DNAT and SNAT.
Ript provides a method to group common sets of rules together called partitions, which are used at rule application time to perform zero-downtime migrations. This fosters a much more agile approach to firewall changes that limits the size and helps increase the frequency of changes - core principles behind Continuous Delivery. Ript is designed from the ground up to be easy to use, and is extremely well tested end-to-end. Developed at Bulletproof Networks, it's been in use since 2012 in multi-tenanted firewall platforms as well as standalone systems.
In this talk Lindsay Holmwood will take you on a whirlwind tour of the DSL, explain how Ript utilises iptables features to work its magic, and provide some concrete examples of how Ript can help increase the reliability of the services you deliver.
Jan Doberstein | The truth is in the logs (DE)
The ability to read and understand log files is very important for a system administrator. But what if, he is so busy in interpreting and analyzing them that he has no time for something else? How can the lazy system administrator hand this task over to somebody else, without providing access to the server?
This talk will show, by means of examples, the evolution of self-written CGI scripts, which worked directly on the log files, to the first data based systems until today, where techniques from Web 2.0 are used.
Falk Stern | Datacenter Management mit Racktables (DE)
Philipp Reisner | Neues in DRBD9 (DE)
DRBD is a block based replication solution, known and available for years under Linux. DRBD allows the implementation of high available systems without SAN. Further use cases are the implementation of storage heads for IP based SANs and long-distance replications over the internet.
At present DRBD has a general limitation on two nodes. With DRBD 9 a new release (beta) is under way, allowing for the first time a replication up to 30 nodes.
With these features many application possibilities open up for cluster file systems, storage for virtual machines “cloud storage”, and much more.
Michael Renner | PostgreSQL im Jahr 2013. Wo sind wir? Wohin geht die Reise? (DE)
In the past years much has happened around the PostgreSQL project. On one hand we added several robust replication mechanisms to the core project; on the other hand a downright exodus towards PostgreSQL began, due to the ongoing precarious sometimes even problematic product and licensing policy of former successful competitors.
In this talk we will have a closer look at the important features and innovations of PostgreSQL 9.1 and 9.2, try to define NoSQL databases, outline the future developments of PostgreSQL, especially with regard to Scale-up and Scale-out.
Kristian Köhntopp | 2000 databases later (ENG)
Late 2012 it became clear that Oracle MySQL would be releasing the 5.6 upgrade RSN. As we already had the beta running productively since summer 2012, it was clear to us that we would need the ability to roll this out widely once it became Generally Available as a stable release.
Unfortunately, at that time the server count was well in the four digits and rising in preparation of the 2013 high season.
So we needed to make the move from automation to orchestration, being able not only to create new database servers without touching them, but also to deal with replication hierarchy capacity as a service: How many servers can we move out of service at once for upgrade, how to we automate load balancer configuration, how to we validate an upgrade, warm up databases and move them back into service without creating load impulse responses or similar problems?
We came up with a solution that involves bittorrent to distribute data inside the data center, and some work queue that orchestrates automated systems maintenance reliably. We were able to do the upgrade technically in less than three days, and then decided not to go actually that fast - but upgrading became a normal operational process instead of being a crisis.
Eric Lippmann | Configuration Management with Verbosy (DE)
Inventory and configuration management belongs to the common business practices of every enterprise. It doesn´t matter if you use commercial products, open source software like i-doit or excel lists – all of them have advantages and disadvantages.
The weak points of this solution are the restricted flexibility due to stipulated patterns, insufficient extendibility and automatisms and user-hostile interfaces.
Verbosy, in contrast, is different – open patterns, inheritance, the web and command-line interface with SQL-related appliance and a plug-in framework, to name some examples. The motivation for this project, the techniques, the functionality and the interfaces will be introduced in this presentation.