TNS Makers: Luke Kanies on the Docker Phenomenon and the Reality of the Enterprise

Luke Kanies, Puppet Labs: The New Stack Interview, Part Two

Alex: What are you seeing as a result of multiple service providers combined with lots of data? I’m seeing more interest in data streaming capabilities and the ability to segment that data on the fly, or do the compute on the fly, like we’re seeing with AWS Lambda and such. What change does that reflect and what does it mean for you?

Luke: To be honest, I don’t track the ultimate cutting edge of the data world these days; I notice the things that drive customer buying patterns. Certainly things like Hadoop have driven customer buying patterns. We’re beginning to see … it’s not fair to call it backlash, but it’s getting close to that. Large companies have been heavily investing in Hadoop for the last couple of years. In a lot of cases it went, “We need big data. Go do something.” They didn’t really understand what the goal was, and they didn’t set hard guidelines for what they were trying to accomplish. So, one or two years on, they look back and they go, “What are we really getting out of this? Is this what we expected? Are we getting what we wanted? Let’s just rethink it a little bit.” Hadoop is the biggest, clear driver of investment in data and doing things differently with data in the last few years that I’ve seen.

Now people are realizing that you can’t just write a check and suddenly become smarter about data.

It’s a bit how, in our field, where companies are realizing they have to become excellent at technology. I think that the lack of standout success around Hadoop has helped people realize that you can’t just write a check and become excellent at data — you have to pay the price to become an expert at data, to deeply understand not just what data you have but how to get the most out of it.

I think that’s going to translate to a lot more niche investments in data. Instead of, “there’s one tool to use” — which is how people have been thinking about it — you’re going to see a lot more diversity in how people can get the most out of the data they have, and how people can best apply the wide variety of interesting new tools around data. I personally don’t know that much about them, though. I don’t track that aspect of the world as closely as other parts.

What other backlashes are you seeing?

Luke: Again, I wouldn’t go so far as saying it’s a backlash against Hadoop. I’d say it’s a realization of how complex the problem really is for people — they’re having to reevaluate some of their thoughts.

OpenStack is certainly interesting because OpenStack has been big for a long time. Somebody said last week, or maybe it was this week, “The market’s beginning to realize that there’s no such thing as a vanilla OpenStack — you have to pick a variant, you have to pick a distro. Once you’ve done that, then that brings with it all these other assumptions and constraints that you have to accept. Because you must pick one or the other you can’t say, ‘I don’t want to make any decisions; I just want the generic one.’ There’s no such thing.” That itself is leading to a bunch of people making different decisions on having to translate their goals and concerns into a concrete world view.

How does that change in viewpoint, about OpenStack, impact Puppet Labs?

Luke: In general, it’s good for us — the last OpenStack community measures showed that about 40 percent of all OpenStack clouds were built and managed by Puppet. It’s not just the specific tech stack you use in your OpenStack installation, but it’s how you install it, how you manage it, how you automate it, how you upgrade it. We’re a big part of that automation story for most of the vendors. Even though we’re not part of OpenStack — people won’t say, “Puppet is the default OpenStack solution” — if you look at how the leaders in the OpenStack community are doing OpenStack, they’re mostly using Puppet. That actually worked out pretty well. The market is going to stop saying, “I’m going to look at the standards,” and they’ve started saying, “I’m going to look at Mirantis, or at Red Hat, or at somebody’s specific distro.” Overall, it’s been good for us, because we’re fantastic at automating complexity, and OpenStack is complexity-meets-automation.

Docker is another movement that everyone talks about.

Luke: Everyone is a big fan of Docker.

I’m not so sure everyone’s a big fan of Docker, though.

Luke: Speaking of backlashes, huh?

Yeah, what’s your view on that?

Luke: The way that too many people have been talking about Docker … it’s just not that easy. People come up to me all the time and say, “With Docker, Puppet’s unnecessary now, right? Puppet’s irrelevant in a world with Docker.” It’s just not that easy — again, this is one of those things that led to that comment,

“There’s no future in which there are fewer servers.”

A) No technology spreads through the enterprise that quickly. I don’t care how awesome it is. You don’t take the millions and millions of workloads that are in production right now and suddenly swap out the virtualization layer for a completely different deployment packaging, runtime, all that stuff. And it’s not because people are evil or lazy — it’s because they’ve got billions and billions of dollars of revenue on the line. They’re going to approach it with some conservatism; it’s going to take time to figure out what it means in practice.

And, B) there’s almost an inherent conflict. If no one’s interested, it gives you the time to focus, to dial in the answer, and to really figure out what you need to get right. If everyone’s interested, on board, involved … then you don’t have that luxury. You don’t get the space; you don’t get the freedom; you don’t get the time.

There’s this inherent trade off. You can get lightning-quick adoption, but when that happens, then suddenly you don’t get the space to really develop and evolve in the way you need to. Or, you can have slow evolution and slow adoption, but you get that space.

Docker went from nothing to “everyone agrees it’s the world’s savior” faster than any open source technology ever has, as far as I know. That’s very different from “it’s the answer to everybody’s problems; it’s got massive financial muscle because it’s driving real revenue.” Mongo became extremely popular very quickly; it was the most widely-adopted NoSQL product. But, Mongo wasn’t doing a hundred-million or a billion dollars in sales.

Look at a company like Uber, that can go super-fast from nothing to generating scads of money. It’s tough to go from nothing to generating scads of money in the open source infrastructure space. It takes a lot longer to shift the enterprise world, to shift the production world, than I think most people were realizing. Again, I think the way they talk about it, it’s like, “That’s bad. It’s because enterprise is lazy or evil in some way.” I think it’s just more complicated than that — it’s just not that easy.

Docker or someone like Docker — in some ways, it doesn’t really matter to us which one it is, but it’s going to be someone like that — is going to be an important part of the future. At the same time, if the exact technology that we know as Docker had come out in 2005, we would never have talked about it. The reality is, VMware has spent the last ten to fifteen years investing heavily in helping the whole market understand and move to virtualization. The shift to virtualization that VMware drove — and made a ton of money off of — was way bigger than Docker, in terms of how big of a shift it’s going to be for the market.

In 2000, everyone was physical; it was physical everything. You can find some small exceptions, but it was pretty much one server, one service. That’s not the case anymore. We’ve got this elasticity that Amazon was able to do, because VMware paid the price of driving virtualization throughout the market. It’s also important to recognize that Docker isn’t the beginning of a trend; Docker is riding a wave that’s been going for a long time. Because you can so easily slot Docker into your virtualizational workflows, it’s much faster to adopt. But, if you’re fitting it into existing, pre-built workflows, then realistically, it’s going to look a lot like something you were already doing. Intrinsically, it can’t be as revolutionary if it’s just taking out one LEGO and putting a different LEGO in.

A number of technology movements are associated with Docker. How is that starting to affect configuration management? Has it had an effect? Are you seeing different ways of approaching technology issues that customers are having? What are some of the impacts it’s having on that market?

Luke: Today it’s much more hype and churn than it is reality. That is to say, it’s not costing me any deals …

If you were going to break down the hype and find that bottom layer of realness, what would you say that is, and how would you say it’s affecting? Or, is it all hype?

Luke: One of the things we at Puppet Labs have done really well is to invest in the direction our customers are moving, in a way that works out well, regardless. OpenStack is a great example. When OpenStack came out, I was there at the initial announcement. I was working with Rick Clark as he was building the whole thing out.

But, we didn’t have three people devoted to OpenStack early on; until my customers show up and I have people calling me asking for help, I’m not going to bother.

As soon as they realized, “Oh, wow — I’ve got complex infrastructure, it’s a distributed system, it’s automation,” they asked, “Who do I go to? I’m going to go to the leader in the space.” They call us up; they work with us.

When we worked with Red Hat and Cisco and Mirantis and another company to build the first Puppet modules to manage OpenStack, that was driven by customer demand. We had a lot of response from the market saying, “This was the first time we could trust to run OpenStack in production in our environment.” That was the right combination of paying attention to what’s going on, but not over-investing until customer demand shows up.

With Docker, we’re going to see the same thing. There is real interest. People are using it in production right now. New Relic, here in Portland — they’re using it in production. When I was at DockerCon last year, they were the only company I could find who was actually using it, and they had pretty much set it up the previous week. There are people using it — it is interesting; it is compelling. But, the reality is: you still have to install your software, you still have to update your software, you still have to configure those containers, and you still have to do things with it.

The intersection of what Docker does really well, and what we do really well, is where your customer is going to go in the market. The specific way they trigger Puppet — maybe instead of starting a hundred machines and then running Puppet on them, they build one machine with Puppet, and then they start a hundred clones of that — that’s different, but it’s not fundamentally different; it’s not a less strategic role in getting the work done.

Again, it’s not rocket science and it sounds like an easy answer. But we work hard, on the one hand, to understand and track the market effectively, but on the other hand, to drive our behavior by customer behavior. Customers are definitely excited about Docker and are doing interesting things with it. But, so far, everything that we’re seeing people do with Docker looks a lot like packaging, or a lot like virtualization, and we work really well in both of those worlds.

Increasingly we’re talking generically about services and software on sophisticated, fast, and distributed infrastructure. How do you see that landscape right now, as companies think less about private cloud versus public versus hybrid, but more about just doing things on that infrastructure itself?

Luke: The big driver is that everyone’s trying to move faster. For a long time, the trend in the enterprise was that we achieved greater stability by slowing down. What the enterprise has realized very clearly now — and this is from the most conservative insurance companies to the most cutting-edge, high-frequency trading organizations — is that stability and security don’t come from moving slowly — they come from moving quickly. Even if you’re scared of moving quickly you don’t have a choice, because the market’s just moving too fast. If you’re not out there driving the leading edge of the market, then your competitors are.

For everyone who has ever worked with Puppet Labs, pretty much the only reason why they’re working with us is they’re trying to find a way to move faster.

They’re trying to find a way to accelerate the cycle times. “How quickly can we ship? How quickly can we update? How little time can there exist between the developer’s keyboard and the users getting value from the products we build?” That pressure from the CIO on down to the frontline sysadmin is driving the investment decisions everywhere.

The major difference between using a container and using a VM is, frankly, a container just starts off a lot faster, and you can pack a lot more onto the machines — which means you don’t have to think so much about how you invest in silicon. So, that’s driving a big investment, because it’s so much faster and there’s so much less friction in changing your status.

To go back to an earlier part of the conversation — converged infrastructure. One of the reasons people like converged infrastructure is, “I need stuff. I need it tomorrow. I’m going to write you a check, and the stuff’s going to show up, and we’re going to plug in a fiber cable and a power cable, and everything’s just going to work.” It used to take months and months and months to get that going. It’s not the straight velocity; it’s acceleration — how much more quickly I can learn. That’s driving pretty much all the change in the market, and especially it’s driving investment in automation.

We did our DevOps survey last year, and our new one for this year is going to come out very soon. We’ve found that companies that are heavily invested in the tooling processes and practices around DevOps deploy thirty times more frequently and, in doing so, they experience one-half the number of errors.

In this movement, you get faster speeds, lower error rates, greater reliability, greater security, and you’re moving faster all at the same time.

That mutually beneficial system is what’s driving everyone’s behavior. The people who feel that speed and who can really see the benefit of it are loving it, and they’re doubling down on it. The people who don’t have it — with the outsourced infrastructure, and the lack of investment in the last ten years in technology and in people — they’re feeling the pain. They’re trying to figure out, “Where can we invest that’s going to get us on the other side of the line?”

You’re in this new office; you’ve just moved in when?

Luke: Middle of November.

What year did you start Puppet?

Luke: 2005. We’re coming up on my ten-year anniversary.

Where was your first office?

Luke: I worked out of my house for five years. This is our fifth office here in Portland, if you don’t count my first home office in Portland. We were in NedSpace above Backspace for a few months — that was our first office. Then we were in the Merchant Hotel Building, and then we were in the General Auto Building, and then we were in the RiverTec building, and now we’re here. This is about 75,000 square feet.

How many people are working here?

Luke: About 80 percent of our 350 are here in Portland, so maybe 270-280.

What did you want this place to speak about Puppet Labs?

Luke: I worked really closely with our architect, THA, on the design. It’s definitely a collaborative project between the two of us.

The major focus of the design was to build a place people wanted to be. I have to work here every day, too. I want to have an office that I want to be in, and that our team wants to come to, and work in together — a place where people can do great work, and want to do great work. It was, as much as possible, to try to build a place people enjoy being in, and it wasn’t really meant to say anything.

A couple of parts of the space are more obviously dedicated to visitors, to speak to the outside world — when you get off the elevator, when you get up to the front desk, when you step into the lobby. Those are more statements to the wider world, and certainly those are important statements. When people come in to interview, they come in like you do; you’re talking to me, and in the walk through the building you come to conclusions about who we are and how we think about the world.

You have a “Lamb Chop” room, and “Star Wars” and “Howdy Doody”?

Luke: Yes, each of our two floors has a quadrant where the rooms are named after puppets from a movie. We’ve got “The Dark Crystal”; we’ve got “Sesame Street”; we’ve got “Mister Rogers’ Neighborhood”; we’ve got “Star Wars,” which does have puppets; Jabba was a puppet, obviously.

We’re clearly not taking ourselves very seriously if our board meetings are in a room called Gobo. I think that kind of irreverence is important in order to have a fun time.

The overlook where we do our all-hands, and our Thursday morning picture meetings, and share it as our cafeteria and kitchen — when you walk in, you certainly see that we are a company that is obsessed with coffee. I am not one of the people who is obsessed with coffee … but there are multiple pour-over stations, three espresso ports, the standard hot pot coffee — a lot of different kinds of coffee.

People talk about how much energy is in the space, and it’s because people want to be here. It’s not beige; it’s not gray; it’s not dark; it’s not boring; there’s bright colors! In my house, we have bright colors on my walls. Why would I want to come to work and have a place where there’s no color?

Here, in this room (Telly), you’ve got this nice, lavender wall that really changes the feel of the space. It’s bright — it reflects off your laptop, off the walls, off the TVs. It’s not just this one splash of color — it spreads through the whole room. It makes me want to be here more.

We’ve got windows cutting through almost all the rooms — this is one of the very few rooms in the building that doesn’t have windows, and that’s because glass reflects audio badly. The fact that you can sit almost anywhere and get a great view outside, get a ton of natural light … in Portland especially, it’s tough to come by natural light.

We get a lot of natural light, but it’s usually when it has a certain hue. Congratulations on the move. Congratulations on the growth of your company, and keep up the great work. We’re proud of you here in Portland.

Luke: Thank you very much Alex, and thanks for coming by.

Thank you.