Will real-time data processing replace batch processing?
At Confluent's user conference, Kafka co-creator Jay Kreps argued that stream processing would eventually supplant traditional methods of batch processing altogether.
Absolutely: Businesses operate in real-time and are looking to move their IT systems to real-time capabilities.
Eventually: Enterprises will adopt technology slowly, so batch processing will be around for several more years.
No way: Stream processing is a niche, and there will always be cases where batch processing is the only option.
DevOps / Tech Life

How to Build a DevOps Engineer in Just 6 Months

The skills that we seek so furiously are skills that we can teach. Mission Cloud explains how it built its own training program.
Jun 1st, 2023 10:43am by
Featued image for: How to Build a DevOps Engineer in Just 6 Months

In “The Rocky Horror Picture Show,” Dr. Frank-N-Furter sings that he can make a man in just seven days. We’re not that good here at Mission Cloud, but we can make a DevOps engineer in just six months.

We have built an intensive, six-month training program that turns recent graduates and career changers into DevOps engineers. For those in the back of the room, I’ll say it again: We can build DevOps engineers in six months.

I’m not talking about front-line engineers who troubleshoot and follow runbooks: I mean engineers who can build infrastructure. Engineers who can code, who understand containerization, who can wiggle their way into a customer’s environment and work with the team to modernize the heck out of it.

This blog post will dive into why we decided to build this program and the steps we took to make the program successful.

Why a Homegrown DevOps Engineer Program Is Necessary

By some estimates, in the United States, there are only about 6,800 trained DevOps engineers, but over 250,000 active DevOps engineer job openings. Needless to say, it is incredibly difficult to find DevOps engineers, but it shouldn’t be.

Cloud technology has been around for a long time, and the requisite skills to build and change cloud infrastructure aren’t a secret. And yet, it’s hard to find schools with robust cloud engineering programs. A few colleges and universities have built certificate programs, and private, for-profit boot camps have sprung up, but none of them can build what Mission Cloud and other cloud consulting companies need: a builder. A strong DevOps engineer with the breadth of skills needed for modern cloud computing.

Current Schools and Boot Camps Aren’t Cutting It

Part of the problem is there are just so many skills. Cloud computing continues to expand. For example, Amazon Web Services (AWS) adds dozens of new services a year, making it incredibly difficult for any one person to keep up, never mind training programs, which can take years to develop.

When we audited the skills of our engineering teams, we found over 200 skills that each engineer needed to have a handle on. We managed to edit it down to around 150 required skills, but that still left us reeling. As the appetite for cloud industries continues, Mission Cloud needs more and more engineers, but how can we find people who have all of those skills?

The Small Talent Pool Isn’t Getting Any Larger

Finding people who can do the work is a challenge. Technical recruiting in cloud engineering is one of the toughest jobs, because there is a limited pool of talented engineers, and it is growing far too slowly for the industry’s needs. Most of the DevOps engineers in the industry learned the trade almost by accident, as their system administration work slowly transitioned into the cloud.

Another problem was that the limited pool of engineers wasn’t very diverse. We couldn’t rely on people falling into cloud as their careers took unexpected turns, and cloud engineering became more diverse on its own. We needed another solution.

A favorite Buddhist saying goes: “When the only hope is a boat, and there is no boat, I will be the boat.” Mission Cloud was trapped on the same shore as the entire cloud industry: burning out the precious few talented engineers and waiting for somebody else to solve the problem.

We were a tiny startup, weaving our way between the behemoth cloud providers. Was it possible for us to be the boat? Could we create the change we wanted to see in the industry and get to that other shore?

And So, the Journey Began

Our options were limited. If engineers were not arising through spontaneous generation, then we needed to build our own. We started small, literally as small as possible: one employee. Someone in sales who wanted to get into the technical side of the house. Could we turn him into an engineer?

It took almost a year, with about a million missteps along the way, but the answer was, ultimately: yes. We sent him through job rotations in different technical departments and sent him different study plans and certification requirements, most of which were a shot in the dark, but he got there in the end. He became an extremely successful pre-sales solutions architect.

So, there we were, with the seed of a really big idea starting to plant. At that point, we’d proved that it could work, but not that it could work at scale. Maybe we just lucked into a secret genius and this was lightning in a bottle, but maybe it wasn’t. Maybe these skills could be taught, and learned, and applied in the right ways through an engineer training program.

When asked how, exactly, our engineers learned DevOps and cloud engineering skills, “I just Googled it,” was the most common answer.

I was, to be terribly transparent, absolutely not the right person to try to build this program. Despite leading the training department, before I joined Mission Cloud I had only the vaguest sense of how the internet worked (Were there cables somewhere? Under the ocean, perhaps?) I was, ahem, of the dial-up generation — my dim understanding of computers began and ended with zeros, ones and the horrible screeching sound of the modem connecting. (I say all this with some shame; my father was a software engineer and tried to incept some of his knowledge into my head but, alas, it never stuck.)

I had one thing going for me though: an absolutely bullish belief that there is no skill in the world that cannot be taught. This belief survived through many conversations with our good-natured engineers, who endured me grilling them on how, exactly, they learned DevOps and cloud engineering skills. “I just Googled it,” was the most common answer, frustrating me to no end. They had learned the skills because they encountered situations they didn’t know how to solve, and they had the desire to solve them. Curiosity and persistence are wonderful personality traits, and certainly made for good engineers, but I could not find the answers I needed.

Our engineers had learned the skills because they encountered situations they didn’t know how to solve, and they had the desire to solve them.

We knew other companies — big companies with lots of resources — had internship programs to try and teach these skills, but they didn’t seem all that successful. We weren’t seeing the market flooded with hundreds of recent internship graduates, so I didn’t think copying other companies was the way to go.

We needed to build something unique, something so closely embedded into the fabric of Mission Cloud that there would be a seamless transition from the training program into full-time engineering roles.

The Search for the Perfect Teacher

I was stuck on the shore again, so I went in search of a boat. I needed someone to build this program: a talented engineer who no longer wanted to engineer, but wanted to teach, and was a good teacher. Teaching, surprisingly, is one of the most difficult skills out there. I learned this the hard way as an English teacher in Peace Corps China.

As a native English speaker, I knew English, but teaching it — breaking it down into its component parts and passing that knowledge along in a structured way, where lessons built on top of each other — was far beyond what my 21-year-old brain was capable of. Teaching is not a skill most people have, and finding a technical teacher —oh boy.

Kelby Enevold

I looked high, I looked low, I answered questions about the airspeed velocity of an unladen swallow, but at last my quest came to an end when I found Kelby Enevold. This former Army communications soldier had become a skilled AWS cloud engineer and trainer, and I was lucky enough to snap him up as my head of technical training programs.

I’m about to get into the meat of what we built, but all this was to introduce the main point: Building technical training programs is hard! It takes a lot of effort and investment. It is incredible that my small company believed in the vision of what we could build so strongly that they were willing to invest in an entirely new role. This is why we have been successful, though: Mission Cloud truly cares about cloud literacy and is willing to put money behind the idea.

Our Formula for a Successful Training Program

Kelby and I spent months building the learning paths, and then it was time to launch. We brought on several overlapping cohorts of interns and put them through the program. Although we’ve made a lot of tweaks to the training paths and skill development, the basic outline of the program remains the same:

We Pay Our Interns a Good Hourly Rate

Even though it takes them several months to produce work for the company, they are working the entire time. Their effort and time deserve remuneration. The days of unpaid internships are ding-dong-dead!

If you are a small company, you may think you can’t afford this, and it’s true — it might be a real stretch. Try to get creative though — perhaps fewer hours with more intense work or a shorter program or finding a community partner that can fund interns through government programs (like LA-Tech or America on Tech). Unpaid internships means limiting opportunities to people with financial means, which totally sucks, and works against expanding diversity in the industry.

The First Part of the Program Is Studying Only

This is because we have set the entrance bar somewhat low: knowledge of Linux, AWS and networking. Interns need dedicated time to beef up their skills before we can let them onto real client work. During this time, interns are expected to work through training paths, gain certifications and get the reps in to practice these new skills. Enevold built a meticulous training path, ensuring interns achieve each skill necessary to go on to the next step. He leads them through Linux Essentials first, then dives into AWS Solutions Architect Associate certification studies.

Interns start with Linux and basics like text editing, they learn about AWS while studying for Solutions Architect Associate, they learn about containers, they learn about git and Infrastructure as Code.

Interns are Embedded into Departments Doing Real Work for Real Clients

After the study period, interns start the intensive shadowing portion of the program. Although the program manager is their main resource, interns become part of the departments they shadow. They are assigned actual support tickets and have tangible responsibilities.

This piece took the longest for us to set up. Department heads were skeptical — give work to untested interns? Slow down client work? Their reservations made sense, but we kept pushing, and gradually champions in the department started to appear. They saw the incredible eagerness of the interns, how they threw themselves into their work, but more importantly, our interns had the skills.

All that studying, all those gorgeous study paths worked. Our interns were not a drag on anyone’s time. Much more quickly than we had even anticipated, they were able to be a productive part of Mission Cloud.

Interns Have at Least Three People to Lean on

Most programs have interns connected with just one person, and that person is responsible for the development of that intern, in addition to their full-time job. That naturally leads to a lot of dead time for the intern and wasted opportunities.

During shadowing, our interns are still managed by the technical program manager, who checks in with them weekly, if not daily, and helps them understand tasks they are struggling with. Interns also meet weekly with the department manager, who monitors their progress, and a mentor, who assigns them tickets and guides them through the daily work. With this triangle of support, interns always have someone to turn to.

The mentors, mostly senior engineers, were delighted to find that working with interns pushed them to improve. “The greatest take-away for me was clarifying and critically assessing my processes, both technical and organizational, because I’d never had to teach them to someone else before,” said Gabe Norton, senior DevOps engineer.

We Focus on the Mindset and Behaviors of Engineers, not Just Technical Skill

Engineering isn’t about pure skill. Just because I can chop up an onion and saute some chicken doesn’t mean I can write a cookbook. Our interns gain skills, but we still have to teach them how to apply a troubleshooting methodology. We place a huge emphasis on hands-on environments. Hands-on exercises really complete the learning loop. Learn the thing, apply the thing, probably break the thing, learn more about the thing!

We also teach our interns how to operate within a larger department. We show them when and how to escalate problems they cannot solve, and how to combat imposter syndrome. When they move to full-time roles, it’s pretty scary to suddenly have the training wheels removed. We support them through that transition and show them how to trust the skills they built.

We Build a Thorough DevOps Foundation

We provide interns with an incredible scope of skill development. They start off looking at our more than 150 skills on the skills matrix with a big ol’ gulp of anxiety, but we guide them through the varied topics. They start with Linux and basics like text editing, they learn about AWS while studying for Solutions Architect Associate, they learn about containers, they learn about git and Infrastructure as Code.

Then they learn about CI/CD pipelines and the fundamentals of Python. They even start learning about container orchestration. Each step along the way includes actual hands-on exercises. Each week we have team meetings and talk about the things that we’ve learned. And, along the way we’re also working on skills like communication, escalation and problem-solving to make sure they can be a fully functional team member.

The Result

This program can work for anyone; we’ve had a former chef, a former Marine and recent college graduates all go through the program, crush it and become full-time DevOps engineers. Right now about two-thirds of our interns meet the bar to become full-time employees; we want to get that up to 80%. What we’re incredibly proud of, though, is our contribution to making cloud industries more inclusive: 75% of our interns are racially diverse and 50% are gender diverse.

Investing in Your People is Worth Every Penny

No matter what industry you are in, the skills that we seek so furiously are skills that we can teach. Think about the fees you spend on recruiters, the time managers and staff put into interviewing, the brutal disappointment when a new hire doesn’t work out, and the process begins all over again. All of these things are accepted parts of corporate team growth, but they don’t have to be. We can build our teams from the ground up, with the exact skills needed to succeed in our companies.

This takes time, hard work, and yes, money, always money, but it will pay off. For Mission Cloud, it is paying off right now, as our interns-turned-employees blossom, pull others up behind them and energize our teams anew.

Kelby Enevold contributed to this article.

Group Created with Sketch.
TNS owner Insight Partners is an investor in: Pragma, Real.
THE NEW STACK UPDATE A newsletter digest of the week’s most important stories & analyses.