The Cloud Native Community Needs to Talk about Testing
KubeCon + CloudNativeCon Europe is fast approaching, and the recently released schedule got us talking.
There’s a ton of fantastic presentations we’re looking forward to, but it struck us that there is only one talk about testing among more than 100 presentations. And this is one of the largest engineer-focused events in the entire cloud native ecosystem!
I decided to do some research to see if this was an anomaly and if testing, whether integration testing, automated testing, continuous testing, unit testing and even manual testing, was more widely covered at previous events.
But it seems to be a trend across the industry. The WTF Is Cloud Native? virtual conference last year had no talks focusing on testing. It wasn’t included in Red Hat’s 2021 Summit, apart from a great talk about how Volkswagen is using virtual testing for automotive parts. Three talks covered testing at KubeCon in Los Angeles last year, and in 2021’s virtual European event, one testing topic came up.
Testing in the Cloud Native Landscape
In fact, when we take a closer look at the Cloud Native Landscape, we see that there’s no category for testing itself. The place where it’s possible to see a few testing tools is inside the Continuous Integration and Delivery category, albeit not all tools there are Kubernetes native.
Taking a closer look, we can see that this category is described as the place for tools that “enable fast and efficient development with embedded quality assurance,” so it could fit in the Testing Tools category, but perhaps it could be helpful for Quality Assurance / Testing to have a category on its own.
We Need to Talk…
So why aren’t we, as cloud native engineers working on global software development teams, talking more about testing? Why aren’t we sharing our successes, failures and hacks with the community in the way that we do for security or orchestration?
Are we stuck in the mindset of thinking about platforms like Kubernetes versus processes like testing but forgetting that specialist-testing automation tools and QA engineers can make all the difference to the process?
Is it because testing isn’t as important as it used to be when we worked in waterfall development cycles? It’s easy now to ship a quick bug fix if something gets missed the first time around.
Or is it simply that people who are experienced at DevOps/site reliability engineering don’t focus on testing?
I spoke to a number of engineers who work with cloud native tools to ask them how they manage their testing pipelines and posed the question:
Why do you think testing is not a very prominent topic in the cloud native community?
Here’s what they had to say:
Mario-Leander Reimer, principal software architect @ QAware GmbH:
“Testing has never been a prominent topic, no matter if we talk to cloud native communities or not. Quality assurance always falls short when short deadlines and budgets or high demand for features hit the teams. The first thing that gets sacrificed is QA activities. In addition to that, if the tech stack and setup gets too complex, the harder it gets to write good and plenty of tests. These are the root causes.”
I also posed the question on r/kubernetes to see what the community thought.
One redditor said:
u/foottuns:
“I work with openshift; when we build or test a framework we use BDD, Golang and godog. Since I started my career in DevOps and infrastructure, the majority of engineers prefer ditching this subject. A lot of people are not comfortable with it or too lazy. But, there’s also the lack of tools and documentation, which makes people behave like this.”
Several commenters argued that testing is tightly coupled to CI/CD, for example:
u/clustersam:
“Testing is a function of automation these days. If you build something, you write tests that belong in the pipeline. If you find QoS problems, you add automated tests to happen in a pipeline. If it’s a browser app that needs synthetic testing, you use Selenium and throw that into the pipeline too after you figured it out the manual way.
You observe the results later, fix the problems, and the pipeline does its job.
That combined with a good promotion to production cycle that also involves people using the lower envs is probably ‘the way.’”
And on the more humorous (or cynical) side:
u/fearphage:
“Laziness and apathy are the first things that come to mind.”
My Hypothesis
After getting feedback from the community, including DevOps and QA engineers, the general consensus I received was that it’s easy to tell that cloud native is a developing field that is still establishing its best practices. We can look into different examples of areas that are still maturing. Not that long ago, we started to hear about DevOps, which brought the concept of shorter and more efficient release cycles, which today feels like a normal standard.
More recently, we saw GitOps following the same tracks, and we are seeing that more teams are using Git to manage their infrastructure. It’s my belief that cloud native testing will soon follow suit, where teams will not see testing as a burden or an extra amount of work that is only “nice to have” but something that is part of the process that will save them a lot of development time.
So, Let’s Change Cloud Native Testing
I’m sure all of you reading this are tech enthusiasts like me and probably have been building and shipping products for quite some time, and I’m also sure many of you noticed that there are major challenges with integration testing on Kubernetes, especially when it comes to configuring tests in your continuous integration/continuous delivery (CI/CD) pipelines to follow a GitOps approach. This is why I think projects like Testkube are game-changers to make testing easier in Kubernetes.
Testkube’s goal is to make testing pipelines less complex so the lives of our DevOps folks become easier and to provide testers with more capabilities to manage tests in a more consistent and centralized way.
It seems that a lot of folks feel these same frustrations. In little more than two months, we’ve seen more than 120k views on our “Intro to Cloud Native Testing” video, which is pretty impressive for a small startup that just started this initiative recently.
If you run apps in Kubernetes and want to improve the way you test them, you may want to give Testkube a spin. Check our installation docs and check if it’s helpful in tackling the challenges of integration testing on Kubernetes.
