In this fourth episode of The New Stack @ Scale podcast, our monthly program sponsored by New Relic that explores the variety of issues that accompany dynamic services and systems, The New Stack founder Alex Williams and co-host Fredric Paul, New Relic’s editor-in-chief, were joined by two guests to discuss container security in the new frontier of scaled-out architectures.
Jim Reno is the chief architect for security at Apcera, and Armon Dadgar is the co-founder and CTO of HashiCorp; both bring an abundance of valuable experience to the discussion, and they offer practical insights that merit attention.
Historically, security has engaged in protecting end points, said Williams, acknowledging that “today there are almost infinite endpoints, and the shift is to providing a deeper data analysis to look for anomalies, and more.”
“There’s a lot more than just the data analysis,” he observed. “We’re seeing the concept of scale and distributed systems changing dramatically.”
Dadgar cited three such changes: “Broad adoption of distributed systems is one aspect of it. There’s a shift away from more traditional monolith systems to more modern microservices architectures. And then, alongside this, the third tide is the shift from machine-based to application-based virtualization.”
Then he summarized the network dilemma. “Where we had communication in a process going through functional boundaries,” said Dadgar, “now we have communication and data flowing over networks between discrete machines. This introduces a whole set of security challenges that simply didn’t exist when we had the traditional monolith approach.”
“Microservices, and service-oriented architectures, in general, are the right approach to scaling complexity, and the network is the right place to be doing this kind of linking. But the security challenge is real and has to be solved, and we can’t just put it underneath the carpet.”
He described the classic InfoSec triad — Confidentiality, Integrity and Availability — to illustrate the inherent risks that container security should address, and everyone conceded that automation will play a greater role in security as the quantity of telemetry continues to outpace our manual capacity to analyze, much less monitor.
In another moment of truth, when Paul asked who in the organization should be responsible for security in an environment that is so reliant on microservices, Reno said that developers may need to own up to some of it.
“One of the problems with security in software that is always been the case, “said Reno, “is the problem of security being considered as an afterthought. A lot of that goes to the developers, and coming from my background in development, I’ll take my share of the blame as well. It doesn’t always get thought of early on.”
“Its easy for developers to say, ‘My job is to get working software out there; we need to deliver value to our customers.’ In a traditional model, there was a certain amount of tolerance for that, because you could say, “Once this stuff gets deployed into a data center there’ll be perimeter security.'”
“In the container world, that’s got to shift, especially since you’re now talking about that developer pulling executable code from many different sources. He’s got to take a little bit of responsibility for what’s now going into that app. It’s not all his code any more. It’s not even open source that he compiled. It’s now binary images that he’s pulling from someone, and you’ve got to know where those come from.”
Security will always be everyone’s responsibility, said Reno, but in this environment, developers may need to make it a greater priority. “One of the things we need to do is give them the tools and systems that help them do that.”
Dadgar said that he likes to point out to clients that security is a spectrum — how much do they want to invest in making something secure?
“The amount of effort you put into security will just increase that cost, but there’s no such thing as a perfectly secure system,” he said. “There’s no finish line that you can get to. You can only make it exponentially more difficult to break the system. Depending on what level of investment you make, it changes who’s involved.”
“In large organizations you have your InfoSEC team, and they represent a really high investment. You’re going to hire security experts that, all day and night, their job is to think about securing the system,” he continued. “The flip side of it is, you might say, ‘That investment just isn’t worth it to us. We value our agility more, or we value time-to-market… and we don’t want to make security our checkpoint.’ That’s a conscious decision you have to make as an organization.”
“If you’re going to say, ‘It’s my developer’s responsibility,'” said Dadgar, “then you’re forcing them to make a trade-off, because their time is finite.”
“I have 8 hours in the day. Am I spending 8 hours a day shipping a new feature? Am I spending 7 hours a day shipping a feature and an hour on security? Or am I spending 7 hours on security, and one hour of the day on a new feature?”
“There’s no escaping it,” he said. “The trade-offs are fundamental and the cost is not possible to sheet.”
To subscribe to The New Stack @ Scale podcast or check out other episodes, visit the podcast section of The New Stack.
Apcera and New Relic are sponsors of The New Stack.