Drone.IO: A Containerized CI/CD System for Cloud Native Development
Brad Rydzewski recalled his initial inspiration for building Drone, which he claims is the first open source container-native CI/CD software: “I wanted to architect out all the red tape companies can put around these tools.”
Rydzewski created Drone while working at a large corporation. He needed an environment to build, test and deploy his code. Unfortunately, the IT department required he had to submit a requisition ticket to get a Jenkins-based environment, which could take the department to six months to provision. It wasn’t the first time he felt this frustration.
“Every time I needed to make a change to that environment, I had to open up a ticket, and had to wait 30 or 60 days. It really slowed down my velocity as a developer,” Rydzewski said. He experienced similar difficulties at another enterprise company he worked at, where it could take 60 days just to get a Jenkins plug-in approved. Another problem–working in a shared environment meant he was bounded to whatever (usually older_ version of Java the rest of the company was using.
Although part of the problem was that the organization structure couldn’t meet a dynamic development environment, it was also true that the Jenkins architecture was limiting as well. “Jenkins is a centrally managed system. It wasn’t really built for this on-demand cloud experience that we as developers are accustomed to,” Rydzewski said.
The idea with Drone is to “empower the developer with a single-click build environment,” he said. Fortunately, he started building it right about the time Docker started gaining traction. Rydzewski found that, with containers, “the developer is really in control and is managing their continuous delivery, while still given the ops team the security and isolation that containers provide.”
In other words, with containers, you bring your own build environment. You don’t have to ask third parties to upgrade your environment. The Drone workspace is bootstrapped from a Docker Volume, and services, such as databases, are created from Docker Compose files. The workspace and services are then available for every step of the pipeline.
Drone provides both continuous integration and continuous delivery (CI/CD), offering direct connects to GitHub, GitLab, BitBucket, and direct connections to cloud providers such as Amazon Web Services and the Google Cloud Platform. Every time a developer commits code or opens a pull request, Drone will take execute a unit test, and an integration test, and even deploy the code to a development, staging or even production environments.
Unlike most other CI/CD pipelines tools, Drone.io can automatically scale up or down to meet the demand, through the Autoscaler feature. Drone monitors a build queue, a queue of pending location, so if there is a spike of developer activity. Drone can automatically scale up the machine, without having developers wait for their builds to finish. When the build volume decreases, Drone can tear down the servers again.
Because it is packaged in containers, Drone is easy to upgrade. And because it is pulled as a static Go language binary, it was quite compact. The pipeline itself is described as a YAML file, so it can versioned. And the whole setup can be run locally or in the cloud in an identical configuration.
An ARMy of Drones
Last month, startup Drone.io expanded to include support for Arm processor architectures. This means both Drone, and the applications it sends through its pipelines, can be based on Arm architectures.
The latest release of the software, Drone 0.9, supports the Armv7-A and Armv8-A chip architectures. In addition, ARM support has been added to many of its plugs, available on the software’s marketplace, including Slack, GitHub, and Docker.
There was a surprising demand for Drone on Arm, Rydzewski admitted. In fact, the community started work on porting Drone to arm even before the company did. “Over the past year, we’ve seen a tremendous amount of demand for Arm,” he said. The DIY Raspberry Pi community was an early group of users, but Arm work on Drone also expanded into the Internet of Things and embedded computing, as well as to artificial intelligence and robotics.
“In many cases, it is a requirement that they build and test their code on Arm devices,” Rydzewski said. Drone is smart enough to send your code to the correct architecture.
It’s a growing market: To date, there have been more than 130 billion Arm-based chips shipped to date, from multiple processor manufacturers (Arm itself only licenses the design). Bare metal cloud service provider Packet is now offering a $100 credit to get started using Drone with Armv8 Server Infrastructure on Packet.
Cloud storage provider ownCloud moved to Drone, after feeling immense frustration with the popular CI/CD tools of the day, Travis CI and Jenkins. With Travis CI Community edition, a cloud service, they had difficulty replicating developer environments, and upgrades were a major issue with Jenkins. And both were too slow.
Moving to Drone allowed OwnCloud to dramatically increase the amount of unit and integrations tests that can be done on new and upgraded features, without increasing the amount of time each job takes — permitting the company to roll out more features while keeping costs steady. The company has scaled up Drone to 75 servers or more during times of increased developer intensity, but also scales down to a handful of servers in quieter times, saving the company considerable money.
EBay also uses Drone, as does The New York Times. In fact, Rydzewski suspects that Drone may be in a large number of Fortune 500 companies, as developers are spinning up their own copies, unbeknownst to the IT department.
Drone falls under an Apache 2 license and is managed by the company in a similar spirit to how MySQL is managed by Oracle. Basic functionality is extended by a range of plug-ins, under a business source license — basically free to companies making $1 million in revenue yearly.