AppSec ‘Worst Practices’ with Tanya Janca
At the recent RSA 2023 conference in San Francisco, Tanya Janca presented an only slightly tongue-in-cheek keynote speech covering fifteen “worst practices” committed by DevOps teams. As she noted in introducing her talk, “Quite often when we read best practices we are told ‘what’ to do, but not the ‘why’. When we are told to ensure there are no false positives in the pipeline, the reason seems obvious, but not every part of DevOps is that intuitive, and not all ‘best practices’ make sense on first blush.”
Janca, who goes by SheHacksPurple, is the best-selling author of ”Alice and Bob Learn Application Security” and the founder of We Hack Purple, an online learning community that revolves around teaching everyone to create secure software. Janca has been coding and working in IT for over 25 years, has won countless awards and has been everywhere from public service to tech giants, writing software, leading communities, founding companies and “securing all the things.”
After her keynote, Janca sat down with Taylor Armerding of Synopsys for an episode of AppSec Decoded where they did a deep dive into four of the most crucial “worst practices” that organizations want to avoid when building a robust DevSecOps program.
Q: What do you mean by runaway testing? And why is it a problem?
So, basically, all the people that do DevOps, they want to go fast. And let’s say I buy a new analysis tool to put in my pipeline. Best practices would be that I make sure to time it, to check that it’s actually fast. Then I need to tune it, adjust it, play with it a lot, so that I can make sure it’s testing what it needs to test and doing that fast. The worst practice, and I’ve seen this happen, is where people just put the tool directly into a release pipeline live. Maybe it works right out of the box, but what’s much more likely is that the test takes like six hours. This means you’ve clogged up your pipeline, and you’re wasting other people’s time because no one can get any work done. You’ve monopolized all the resources by not testing your tools first. This can make you really unpopular.
Q: Why is only worrying about your part a “worst practice”?
This isn’t necessarily a technical thing. It applies to just about any workplace. I used to be a software developer, and I would kind of just throw my app over the wall to Ops and be like, “Have fun with that! Works on my machine!” but in reality, DevOps is all about emphasizing the speed of the entire system, and not just your part. Unfortunately, developers experience security as someone who comes in like three-quarters of the way through a project and says, “By the way, over the next two weeks we’re testing this.” If you’re a developer, this delay is not on your schedule, and you still have to make your deadlines. So, security becomes something that you see as impeding your work. To get to true DevSecOps, it’s really really important that we actually worry about other people’s parts in the development cycle and how our parts fit into their activities so we can work together.
Q: What makes hiding mistakes and errors a “worst practice”?
So this is a hard one for me. Someone had to teach this lesson to me, and while I’d like to pretend I’ve never done any of the 15 worst practices, this is the one that was the hardest lesson. I was having a lot of trouble getting developer buy-in from this one team. The manager told me I was wasting everyone’s time with “that security stuff” and so my boss said, “We have to connect with this team.” We’d had a giant security incident with one of their apps, so he called a closed-door Chatham House Rules meeting with them. It was an SQL injection incident, and we found out our data was for sale on the dark web for $50! First of all, that made us feel bad because we’re worth more than $50, and even worse, we found out when the media wanted a comment. My boss is telling them this and the team is being sympathetic, and that’s when he tells them, “That was your app and Tanya tried to test it, and you wouldn’t let her. It cost us hundreds of thousands of dollars and who knows what it’s going to do to our reputation because you won’t listen.” I was just like redder than my dress because I felt like I made a mistake, and that’s when they started talking to me. They wound up vowing that this will never happen on our watch ever again. From then on, they were my No. 1 champions. So that’s how I learned to share when I screw up, and learned to say, “Hey, I’m having trouble with this.”
Q: Which “worst practice” do you want to expand on?
Impossible SLAs. Having reasonable service-level agreements is so important. When I work with enterprise clients, they already have tons of software that’s in production doing its thing, but they’re also building and updating new stuff. So I have two service-level agreements and one is the crap that was here when I got here and the other stuff is all the beautiful stuff we’re making now. So I’ll set up my tools so that you can have a low vulnerability, but if it’s medium or above, it’s not going to production if it’s new. But all the stuff that was there when I scanned for the first time, we’re going to do a slower service-level agreement. That way we can chip away at our technical debt. The first time I came up with parallel SLAs was when this team lead asked, “Am I going to get fired because we have a lot of technical debt, and it would literally take us a whole year just to do the updates from the little software compositiony thing you were doing.” “No one’s getting fired!” I said. So that’s how we came up with the parallel SLAs so we could pay legacy technical debt down slowly like a student loan versus handling new development like credit card debt that gets paid every single month. There’s no running a ticket on the credit card!
Q: What’s at the root of these “worst practices”?
One day when I was early in my career as a developer, a security person ran a VA scanner on my app. When I asked how do I fix these errors, he just said, “You should know.” I was panicked. How should I know? I’d had zero training on secure practices. They didn’t talk about this in college. No one ever gave me a book. None of my colleagues spoke about this. What do you mean, I should know? I learned that actually he had no idea what the answer was, and he felt insecure, so he did the “blame her and shame” so I wouldn’t ask again. Most people get into development because they want to build an amazing app that delights their customers, that does all the things they asked for, and they’re doing it the way they’ve been taught. What we need to do as an industry is start sharing from the very first lesson how to do things securely
AppSec Decoded Can Help
AppSec Decoded is a regular production of the Synopsys Software Integrity Group. You can access videos on security topics and interviews with key players in this space on its YouTube channel.
