CI/CD / Development / Tools

KernelCI Joins Linux Foundation to Accelerate Linux Kernel Hardware Testing

11 Nov 2019 7:25am, by

Testing software can present difficulties enough on its own, but when you’re talking about the Linux kernel — that piece of software that directly interfaces with hardware — there are innumerable more complexities added to the scenario. One of those complexities, of course, is the number of devices Linux is intended to run on, from a variety of desktops, laptops and phones, to Raspberry Pis and other devices on the edge. Each device presents a new environment in which the Linux kernel needs to be tested, and that’s where KernelCI comes into the equation.

KernelCI is an open source, distributed test automation system focused on upstream Linux kernel development that has just recently joined the Linux Foundation with backing from BayLibre, Civil Infrastructure Platform, Collabora,, Google, Microsoft and Red Hat. The project, which was first started in 2014 with a stated mission “to detect, bisect, report and fix regressions on upstream Kernel trees before they even reach «mainline»,” currently tests the Linux kernel against nearly 250 unique hardware platforms, but by joining the Linux Foundation it hopes to expand this number further.

“Testing is traditionally done only on the most common hardware. But because Linux runs on more hardware than any other operating system, it’s important to also test it on all that hardware,” explained Kevin Hilman, co-founder of embedded Linux consultancy BayLibre and co-founder of the KernelCI project, in a statement. “The Linux Foundation’s support is enabling us to expand the great work we started five years ago and sets us up for a bright future with a growing community.”

Linux Kernel Testing Across a Variety of Hardware

In an interview with The New Stack, Hilman described the project’s origins as a bit of a grassroots effort from both Linux kernel developers and the ARM community, which comprises a variety of embedded system devices, such as Raspberry Pi.

“It was born out of the Linux kernel developer community for the developer community and then over the years it grew and we started getting requests for more features and to test more parts of the kernel and to add more hardware and build up the labs. Slowly, we added a bunch of hardware in distributed board farms around the world, and that’s kind of where we are today,” said Hilman. “The Raspberry Pi is one example of thousands of different kinds of single-board computers that have ARM processors that run Linux. And so, those of us who were maintaining the Linux kernel for that type of system, we just have this massive variety of hardware that we want to make sure the Linux kernel still works on.”

In its current incarnation, KernelCI monitors around 100 Git trees, and new Git trees can be added to KernelCI’s testing process through a manual email request. But Hilman says that the future focus for KernelCI is on both expanding the variety of hardware as well as potentially including other types of software beyond the stable upstream Linux kernel, such as Fedora and Debian. In addition to an expansion of KernelCI’s capabilities, Hilman says that he hopes that by joining the Linux Foundation, KernelCI will be able to consolidate with other projects in the Linux kernel testing realm.

As for the “next generation of KernelCI,” Hilman says he hopes to take advantage of some of the designs and tooling around microservices architecture.

“If you think of a Raspberry Pi sitting in somebody’s closet somewhere booting Linux every once in a while and spitting out logs like a microservice — all these little boards that are coming up and down running tests — there’s a lot of this distributed cloud tooling and log analysis stuff that’s actually quite appropriate to this problem,” said Hilman. “So I’ve been thinking about, as we evolve the project, how do we take advantage of this kind of container management and distributed log capturing, collection and analysis stuff that’s actually quite relevant to what we do in KernelCI.”

The Linux Foundation is a sponsor of The New Stack.

Feature image via Pixabay.

A newsletter digest of the week’s most important stories & analyses.