Kubernetes is not your friend
Kubernetes. There is so much noise about this system in the industry, from leaders to devops and developers. Everyone seems to be excited about it and there is a promise that Kubernetes will solve all your problems. But what is K8? The official website says:
Kubernetes (K8s) is an open-source system for automating deployment, scaling, and management of containerized applications.
What the heck does it mean? Literally what it says here - K8 is designed to help with container deployments, upgrades, and scaling.
All about containers
K8 only works with containers, and only with Docker containers. It helps you to host them. K8 was not the first one to do that, there are many other players on the market. There is Docker's own Docker Swarm, Marathon, Amazon ECS, Azure Container Instances, HashiCorp Nomad and so on. And those are just for running Docker containers only. But why everyone shouts about K8? It's just because it became the most popular, the most reachable and has the biggest community. Does it mean it's the best on the market?
It's about cloud portability
K8 is the most popular and the most accessible container orchestrator, and is offered by all the major cloud providers and more, for instance:
- Azure Kubernetes Service from Microsoft
- Google Kubernetes Engine
- Managed K8 from Amazon - Amazon EKS
What this means is that at least in theory you are not tied to a particular cloud vendor to develop your applications, provided that they are containerised. In theory.
It's not portable between clouds
Cloud portability is a big myth and it just doesn't work. What K8 promises is that you can just take your containers and run anywhere, on premise or in any cloud. And this is absolutely true. It works very well in practice too, provided that:
- Your application doesn't store any data
- It doesn't do anything with a network, at least it doesn't accept any incoming traffic
- It doesn't require any monitoring
- It doesn't use any other PaaS services
- Fully stateless, has no interaction with anything outside of itself
And those are just the most obvious points. I haven't seen many applications that don't need any of those, maybe only the ones in starter tutorials or tech talks.
This is not a development framework
Against popular impressions that K8 is needed by developers, it's actually not true. K8 has to do nothing with code or application, it's a pure infrastructure automation solution. There is nothing to code specifically for K8 or anything to take into account. K8 runs containers, that's about it. It's a DevOps solution.
You don't need any special tools to develop for K8, locally or remotely. As long as your application can be packages as a container it doesn't really matter. There is nothing you can use in code to take advantage of K8 services. There are no services in K8 for applications to use. Are you a K8 developer? Bullsh*it, there is no such thing.
Setting it up is a pain in a bum
Setting up K8 is a real pain. Especially if you are installing it on premise. Of course the set up procedure is scripted and easy, but what you will get out of the box is just a starter template you need to work on for a long time to add services that can actually support your application(s) in real world production scenarios. This is why K8 is usually used in the cloud, when a large company provides a managed instance for you. Just because setting it up yourself is a massive overcomplicated pain. And you can easily screw things up bringing down your whole cluster.
It's designed for small workloads
Surprised? Yes, K8 is not a distributed system. Funnily enough, it's a centralised, slow and not designed to scale. It's a heavily centralised system. It has API server in the middle, and agents called Kubelets installed on all worker nodes. All Kubeletes talk to API server all the time for various administration reasons. Apparently, the more worker nodes you have, the more traffic and pressure is required for a cluster to operate normally. Of course you can increase the amount of API server nodes, but that has certain limits and they are pretty easy to reach.
Deployment is hard
In order to deploy anything to K8, you need to create a deployment resource. This is somewhat large and clunky yaml file. In practice, no one is using native K8 features for deploying services, because of it's rather ugly limitations:
- There is no templating (i.e. using simple environment dependent variables)
- No way to describe dependencies on other services
- No way to describe an application, you can only describe a single service, which is not that useful.
To overcome those limitations, people have started creates systems on top of K8 just to deal with simple deployments, and there are so many of them:
- Helm package manager, which is one of the most popular ones. It's actually beautiful on the surface, however is a mix of yaml and go templating languages and quickly grows into an unreadable nightmare.
- GitKube
- Kapitan
- Kustomize
- Kontemplate
- and so on...
The fact the amount of these "tools" are growing and that companies are creating their own custom tools to do similar job, because built-in just don't deliver, doesn't make me trust this system very much.
But it's an industry standard
This is probably the major argument towards K8. It's open, understood (somewhat) by many, and isn't owned by a single company. I'm not sure this is a good argument though when it comes to success.
Data Storage is non existent
You can't store data in K8, there is simply no way to do it. It's not supported at all. Support for volumes is laughable, it's just a way to emulate a disk by mounting it to a remote storage over a network. Yes you can mount a folder to something like Azure File Share or S3 bucket, but performance will be extremely bad. If you run on premise, you can probably use some sort of physical disk, but it's a huge pain to set it all up and make it reliable.
The only way to store data is to use external databases, but those have to run somewhere as well. And you can't run them in Kubernetes, because there is no storage support, so you have to rely on external providers running on bare metal or PaaS services (SQL Server, DynamoDB etc.) which again doesn't make it self-sufficient and cloud portable.
You need an army to support it
It doesn't come cheap and can't be just used by developers.
What is it good for?
To be honest, I have no idea. Can you think of a real use case?
If I need to write a stateless service, that is scalable, and I do care about productivity, price and delivering business goals, I'd go with serveless functions (AWS Lambda, Azure Functions etc).
If I need to process large amounts of data in a long running background process, I'd go for batch solutions like Azure Batch or AWS Batch - no infrastructure management and much lower cost.
If I need to develop a service that needs to work with a lot of data and performance is critical, go for Service Fabric, which by the way can run Docker containers too, amongst many other things.
Hosting a web site? I'd be crazy to host this in K8, because PaaS solutions like Azure App Service are next century tech in terms of features comparing to K8 where you need to build everything yourself for another year.
Do I need to host containers? If so, Azure Container Instance or alternative from other providers is a way forward.
Thanks for reading. If you would like to follow up with future posts please subscribe to my rss feed and/or follow me on twitter.