Select Page

NETWAYS Blog

Announcing Kubernetes v1.24 and v1.25

We’d finally like to announce the release of Kubernetes v1.24 and v1.25 on our Kubernetes Platform. Since 1.24 brought many under the hood changes, our deployment process had to be refactored as well. While Version 1.24 and 1.25 were available on our platform for some time now, we can now safely say that both versions are completely stable and safe to use. For the various changes coming with the new version we recommend creating a test cluster to test your applications. But what are these big changes? Let’s go through some of the highlights.

Deprecation of the docker shim

Easily the biggest change is the deprecation of the docker shim that broke our current deployment method where every component and workload runs in Docker containers. But with this release the docker integration is no more. Kubernetes now officially only supports Container Runtimes implementing the CRI specification. Container Runtimes that implement said spec would be Containerd, CRI-O, Kata and gVisor for example.

Since Containerd is the most common runtime, we settled for it. Even though docker internally uses Containerd, the migration was pretty tricky, as it involved installing and configuring Containerd, as well as making sure that Kubernetes uses it. Finally when kubelet restarts, all containers will be recreated.

Containerd

We initially thought Containerd would behave exactly like docker since Containerd runs the docker containers after all. But for one reason or another that is not the case. Through the CRI interface the containers get created based on a CRI-spec. Unfortunately, the default configuration of Containerd does not set the ulimit properly, which results in some application working and others will be killed by the OOMKiller. It turns out that some application try to check the ulimit by “trial and error” and since the limit is set too high, the kernel will eventually kill the respective process. Weirdly enough, almost exclusively older applications were effected. For example mysql:8 would work fine, but mysql:5.7 will crash almost instantly. The same problem can be observed with the nfs-server-provisioner and rabbitmq for example.

API Deprecations

Like with any other Kubernetes release, there were a lot of API deprecations that needed us to move the Flannel CNI and other deployments to a new release gracefully. The official Kubernetes Blog has a great write-up on this topic.

Configuration

Another problem we faced is the change in configuration. The core Kubernetes component kubelet now only supports being configured with a special configuration file, which in turn meant that we had to rebuild the configuration from scratch, as all of our configuration involved command line flags that now no longer work. CoreDNS as well had some new configuration options helping to conquer overloading the pod with many concurrent DNS queries. We even support adding new static host entries in CoreDNS. The ConfigMap coredns-extra-hosts sets the entries. This entry hosts.list is empty by default, but can the modified like any other hosts file ( 1.2.3.4 example.com ). After restarting the coredns deployment the host can be queried.

Deprecation of PodSecurityPolicy

PodSecurityPolicies have been deprecated since 1.21. But since 1.24 is the last release with it still active, it’s the last chance to get started with PodSecurityAdmission. However, they don’t provide the same feature set, as it enforces the policy based on 3 Pod Security Standards namespace wide. This means, in order to get the same and even more features solutions like OPA Gatekeeper or Kyverno have to be implemented.

ServiceAccount Tokens

Another noteworthy change is the changed behaviour in token creation. Until now, every ServiceAccount automatically gets a new secret including a token to access the Kubernetes api. This is no longer the case. If you need to create a token, make sure to use the kubectl create token command instead.

Happy upgrading!

Justin Lamp
Justin Lamp
Systems Engineer

Justin hat 2022 die Ausbildung zum Fachinformatiker für Systemintegration im "echten" Norden abgeschlossen. Durch seine große Verbundundenheit zu Open Source hat er aber schnell gemerkt, dass ihm Themen im Kubernetes und OpenStack Bereich mehr liegen als im propreritären Microsoft/ VMWare Umfeld. So hat er beschlossen den Schritt zu wagen und andere Teile Deutschlands zu erkunden, um NETWAYS im Team Web Services tatkräftig zu unterstützen. Wenn er nicht in den Untiefen des Linux-Universum unterwegs ist, macht er leidenschaftlich Leichtathletik, geht Wandern und Mountainbiking.

NWS goes stackconf

About stackconf 2022

This year’s stackconf takes place in the beautiful Meliá Berlin. After a long draught, the conference can finally take place in person! We can finally meet other developers, users and people interested in the same topics again.
The conference features talks about a wide variety of topics related to cloud native technologies, infrastructure automation and lessions learned.

 

Who is going?

Being a silver sponsor most of the NWS team is present at the conference. We would love to get in touch with you and talk tech! in addition our CEO, Bernd Erk, is going to give the opening speech and Sebastian Saemann will show us how to use Vitess on Kubernetes with an intriguing demo.

 

What am I personally looking forward to?

I personally can’t wait to get insights into the big Spotify outage in March, which I was effected by as well. It’s always great to hear from other companies how they handle failures of any kind. Another talk I can’t wait to see is the Metal³ talk about provisioning infrastructure directly with Kubernetes. Since many workloads seem to shift to Kubernetes it’s a natural progression to manage everything with it. Unfortunately that means that I will miss the equally interesting talk about Nomad, an alternative to Kubernetes, and Waypoint. But since every talk will be recorded and uploaded onto the NETWAYS YouTube Channel, I’ll definitely catch up on it!

 

What to expect?

There will be people from all over the world joining the conference. It will be an opportunity to meet like-minded, interesting people and to exchange opinions, discuss experiences or just enjoy a chat with them.
The schedule already reveals that there will be many top engineers, CTOs and developers from all sorts of companies giving insights into their business, as well as experiences, best practices and invaluable knowledge learned by doing the wrong thing. A lot of interesting topics will be covered – there is surely something for everyone!

 

Are you going?

I’m already excited to go there and can’t wait to learn many new things regarding Kubernetes, CI/CD and much more.
Me and my colleagues are also thrilled that we can join the conference in person! Having the chance to meet other people face to face is just something different and also watching keynotes and demos (fail) live on site is always exciting. Will you be there, too? We sure hope so – otherwise, you can still get your ticket(s) for the on-site event in Berlin!

Follow us on twitter to get some first hand impressions of our experience at the stackconf 2022!

Justin Lamp
Justin Lamp
Systems Engineer

Justin hat 2022 die Ausbildung zum Fachinformatiker für Systemintegration im "echten" Norden abgeschlossen. Durch seine große Verbundundenheit zu Open Source hat er aber schnell gemerkt, dass ihm Themen im Kubernetes und OpenStack Bereich mehr liegen als im propreritären Microsoft/ VMWare Umfeld. So hat er beschlossen den Schritt zu wagen und andere Teile Deutschlands zu erkunden, um NETWAYS im Team Web Services tatkräftig zu unterstützen. Wenn er nicht in den Untiefen des Linux-Universum unterwegs ist, macht er leidenschaftlich Leichtathletik, geht Wandern und Mountainbiking.

NETWAYS stellt sich vor – Justin Lamp

This entry is part 38 of 62 in the series NETWAYS stellt sich vor

Justin Lamp

 

Name: Justin Lamp

Alter: 22

Position bei NETWAYS: Systems Engineer

Bei NETWAYS seit: April 2022

 

 

Wo kommst Du her?

Ich komme aus dem Norden, dem schönen Kiel. Dort bin ich aufgewachsen, und ich habe mein Abitur und die Ausbildung zum Fachinformatiker dort gemacht.

 

Warum NETWAYS in Nürnberg?

Gute Frage, ich habe eine große Faszination für OpenSource Themen und so habe ich schon früh angefangen mit Linux, Docker, Kubernetes, OpenStack und anderen coolen Technologien zu experimentieren.
Daher habe ich mich entschlossen meinen Aubildungsbetrieb nach der Ausbildung zu verlassen und mich voll auf meine Stärken zu konzentrieren. Leider sind im Norden viele Unternehmen im Microsoft-Universum unterwegs, weshalb ich mein Suchspektrum ein wenig erweitern musste.
Da fiel mir dann wieder ein, dass da doch mal so ein cooles Unternehmen mit vielen YouTube Videos zu Kubernetes und OpenStack war.
Tja, als ich die Website dann wiedergefunden habe, stand der Entschluss fest! Zudem bin ich dann näher an den Alpen, wo ich sowieso gerne im Sommer, wie im Winter hinfahre! 😉

 

Was macht Dir an Deiner Arbeit am meisten Spaß?

Am meisten Spaß macht mir, neue Technologien auszuprobieren, sie zu studieren, einzurichten, Fehler zu bauen und diese dann zu fixen. Dieser gesamte Prozess ist einfach super! Da ich ja aus der properitären Welt komme, ist es einfach mega befreiend, direkt mit den Technologien (und wenn nötig derer Entwickler 😉 ) zu interagieren.
Durch die ganzen Vorschriften und Abstraktion bei properitären System, kommt nie ein Gefühl auf, ein Teil des Ganzen zu sein. Man ist immer der Anwender, der im Problemfall bei der Hotline anruft und ein Ticket aufmacht.

 

Was machst Du, wenn Du nicht bei NETWAYS bist?

Ich bin hier in Nürnberg noch in der Orientierungsphase, aber in Kiel habe ich Leichtathletik gemacht, bin häufig in der Ostsee Schwimmen gewesen und bin viel geradelt.
Außerdem bin ich gerne in den Bergen, wo ich dann Mountainbiking, Rafting oder Canyoning betreibe. Wenn dann im Winter Schnee liegt, fahre ich sehr sehr gerne Snowboard. Ich kann schon jetzt die nächste Saison kaum abwarten!

Justin Lamp
Justin Lamp
Systems Engineer

Justin hat 2022 die Ausbildung zum Fachinformatiker für Systemintegration im "echten" Norden abgeschlossen. Durch seine große Verbundundenheit zu Open Source hat er aber schnell gemerkt, dass ihm Themen im Kubernetes und OpenStack Bereich mehr liegen als im propreritären Microsoft/ VMWare Umfeld. So hat er beschlossen den Schritt zu wagen und andere Teile Deutschlands zu erkunden, um NETWAYS im Team Web Services tatkräftig zu unterstützen. Wenn er nicht in den Untiefen des Linux-Universum unterwegs ist, macht er leidenschaftlich Leichtathletik, geht Wandern und Mountainbiking.