In part 1 of this series, I covered the history of security in the early days of the cloud revolution. Circa 2014, DevOps practices started to be adopted by security teams, resulting in the era of DevSecOps. We’ll cover this concept, the dreams of so many, and the reality that ensued.
Security is a laggard
One pattern that I have observed about security teams is they are slow to change. Security was not a forethought in the pre-Internet days, because it was never really a problem. These teams were only created after internet adoption started becoming more widespread, and they integrated into the waterfall processes of the early days - because that is how development teams worked with infrastructure teams.
After cloud infrastructure became available, DevOps as a practice started to spread like wildfire as we covered in part 1. Why did it take security teams 10 years to fully adopt agile practices and integrate security into DevOps?
- Traditional security focused on building a strong perimeter for a static environment. While the rapid iteration and change of DevOps was a boon to development teams, it was naturally uncomfortable for security teams.
- New tools and processes meant new attack vectors: something security teams are naturally wary of
- Security teams were not incentivized to move quickly, given they were a standalone, siloed organization
- They started first by trying to apply legacy processes such as waterfall-style review and approval, to an agile methodology - and this failed
From DevOps to DevSecOps: the dream
By ~2015, it had become apparent that security had lost their gate. As DevOps became the de facto standard, established security teams started losing political capital as they impacted development lifecycles by resisting change. As challenger companies saw success by shipping quickly and cheaply, it became obvious that established security waterfall style could not continue. VC money poured into challenger tech companies that applied the same concepts to security that they were applying to development: shifting decision-making and implementation onto the engineer.
DevSecOps appears to have been coined around 2014, usually attributed to Shannon Lietz of Intuit. The promise of DevSecOps was to further speed development lifecycles without sacrificing security, by embedding security choices into DevOps (hence DevSecOps). Security and software leaders thought they could:
- Save money on security teams by turning them into guideline providers
- Get security for free from developer teams, who would make practical choices to enforce guidelines
- Proactively support developers with automation
However, the dream of DevSecOps ignores some realities that make it a failed practice.
DevSecOps: The reality of today
As DevSecOps was adopted throughout the late ‘10s and early ‘20s, it became clear that the benefits did not materialize as promised. Don’t get me wrong, there have been some improvements:
- Some integration into change management: CI process that can enforce security reviews before deployment happens
- Multi-environment support: Build pipelines can progress the same build through environments
This marginal progress isn’t enough. There are too many vulnerabilities, too many issues, and not enough time for the manual world of security that we’ve built for ourselves. It turns out that pushing security into DevOps processes sucks for all of those involved. Security teams are overwhelmed, spending all their time measuring faults and making lists of vulnerabilities. Devs are spending unnecessary cycles on security config, when they want to do actual development.
We’ve been left with the worst of both worlds with DevSecOps of today. Why is that?
Devs aren’t security or infra experts
One of the primary assumptions of DevSecOps was that developers would be able to make security and infrastructure-related decisions that were correct. Turns out that in practice, this is not the case. Why? Developers aren’t security or infrastructure experts, and they don’t have the time to become experts. Instead of addressing the fundamental problem, we try to up-skill our developers on security and force training hell onto everybody involved.
Consider a scenario where a developer wants to deploy using an Azure VM and write from that VM to blob storage, and a security team stipulates that it should only be deployed within a company’s VPC. Only an experienced Azure infrastructure expert would be able to configure this in a time-efficient way: this requires 5+ different resources, and if you’re using something like Terraform it requires even more expertise!
Automation has failed us
Another key expectation by early DevSecOps experts was that automation would make up for developer time. Platform teams started to emerge in the late ‘10s and early ‘20s, providing tools for developers to move faster. To date, security automation is primarily a reactive function: we still expect configuration decisions to be made by developers, and we serve them a list of issues to fix after the fact.
Worst of both worlds
DevSecOps has evolved into the worst of both worlds. With developers that aren’t experts in security or infrastructure, and a lack of automation, we now have:
- Rampant misconfiguration
- Frequent incidents and outages
- Constant triage of issues
- Reactive posture management, resulting in huge backlogs of security issues
- Overwhelmed developers who don’t want to engage with security
- A lack of clear ownership means when there’s a problem, it is unclear who needs to act
This means the Sec in DevSecOps hasn’t kept up its end of the bargain. We’ve only marginally improved (and that’s debatable), and we’re only incrementally more secure.
The answer? Secure-by-default
In part 3, we will cover the future: secure-by-default. New tools and technologies mean that we can take the burden of security and infrastructure configuration off developers’ plates, allowing software teams to ship quickly and companies to preserve security and reliability.
