APPSEC
CI/CD Pipeline Security vs. IDE plugins vs. Pipelineless Security
November 27, 2023
There is a philosophical debate in Application Security, whether scanning should occur in the CI/CD pipeline or in an IDE plugin on the developer’s local environment. In this blog, we explore the benefits and drawbacks of both AppSec philosophies as well as present a third option: pipelineless security.
Just because we’ve always done something one way, doesn’t mean that’s the best way to do it! TSA PreCheck > taking off your shoes in the airport. Espresso and pour over > Folgers. Most of us know this to be true, yet consciously moving away from “the way things have always been done” can be very difficult. It’s why people often suggest that a disruptive product needs to be 10x better to displace an incumbent.
In our world of Application Security, I often hear from security teams who are looking for alternatives: “We have our security scanners in Jenkins, because that is where scanners go. But there must be a better way.” Well, we think it is time to challenge the status quo in AppSec and examine a new approach that might just be 10x better.
As continuous integration, continuous delivery (CI/CD) pipelines gained massive popularity during the 2010s, AppSec teams were presented with (at that point in time) what seemed to be a natural place to inject security scanning: as soon as code is merged into development or production. With everyone having moved to CI/CD pipelines, why not shift security into pipelines where our developers are? Seems logical enough.
There have been several impressive security companies to leverage the momentum of CI/CD adoption such as Checkmarx, Varacode, and others. Even some newer companies and open-source scanners, such as the secrets scanner GitLeaks, introduced command line interfaces (CLI) to trigger various scans.
And yet, many AppSec teams are left wanting something better. Let’s examine the benefits and the drawbacks of using an in-pipeline security scanner.
Consistent guardrails. Scans can consistently run based on pre-defined steps and results. For example, a GitHub Action can be called to scan vulnerabilities in source code and fail the deployment if it meets certain criteria, as having high or critical severity vulnerabilities in the scanned git branch.
Effective for dynamic testing. Smoke, sanity, and integration tests need to be executed against a deployed environment. Similarly, dynamic/interactive application security testing (DAST/IAST) is expected to be executed against such environment to identify vulnerabilities in the observed data in the operational environment. One product that I like is Oligo Security, which may run a software composition analysis (SCA) scan and reduce false positives by identifying vulnerabilities based on the third-party packages loaded in memory (I’ll skip chasing a squirl that highlights the differences, but I’ll just say that both source code based SCA, and this dynamic approach complement each other).
Code coverage. By integrating security into the CI/CD pipeline, teams face a choice: expend tremendous time and energy deploying and maintaining security configurations in every pipeline regardless of its business importance, or – do what most teams do – determine which pipelines are most critical and deploy security scanners in those pipelines at the expense of 100% code coverage. Additionally, most organizations need to chase newly created repositories, as they might not have the expected consistent guardrails.
As a side note, organizations using mono-repositories (i.e. one repository for all deployed services) have similar challenges, as CI/CD pipelined tend to be triggered based on specific conditions that might be missed when code changes are introduced.
Shame & blame. The common developer behavior is to respond “I’m on it” when their name appears next to a failed pipeline execution, especially if it blocks others from pushing code to production. Developers don’t like to see their names next to a “red” pipeline execution result. Scanners in CI/CD are violating the golden rule of “praise in public, criticize in private.” In fact, in-pipeline security scanners criticize in public, and praise... never.
“Shift left” is a term (officially in “buzz word” territory) used in security to describe the movement of security controls toward the developer – or “left” in the software development lifecycle. Snyk was founded in 2015 as “the developer security company” and could not have timed that positioning statement better. Snyk understood the importance of developer experience, and therefore, pushed the scanners all the way into the developer’s local environment – specifically in their integrated development environment (IDE). “Go where our developers are” ... also logical.
While “shift left” is well intentioned, it seems that shifting security all the way to the IDE also shifted some unintended consequences for both security and development teams.
Low privileges required for setup. Because the IDE is in the developer's local environment, they require minimal privileges to set up.
Immediate feedback loop Developers that use the security plugins in the IDE are provided all security feedback directly and in real time as they are writing code. Though as we will discuss shortly, there is another side of this coin.
Partial code coverage. We all know that every developer is a unique and beautiful snowflake. They have their own language preferences, their own style, and they have very strong opinions on their preferred tools. This is a problem for security plugins for IDEs because even if every developer was keen to use the security plugin, if their preferred IDE is not supported, your security coverage has a gap.
Easy to bypass. Saying “if every developer was keen to use the security plugin” contains a very very big “if.” Plenty of AppSec teams are looking for alternate solutions to the IDE security plugin. This is because while dev teams may say they love the IDE plugin, AppSec teams are finding that is primarily because they are often not using it.
Alert fatigue. The “other side” of the immediate feedback loop coin mentioned above is that IDEs are notoriously bad at determining what feedback is relevant. The main question developers are asking is, “what is critical to fix in order to successfully deploy to production?” As the saying goes, when everything is treated as critical, nothing is treated as critical.
Unknown impact. A question you should ask yourself is “how many vulnerabilities were resolved by having an IDE plugin?” The answer I get 99% of the time is “I don’t know.” If you have a similar answer, how can you justify the value of the IDE integration aside from the claim “developers like it.”
At Arnica, we’ve introduced the first ever real-time Pipelineless Security solution. The pipelineless approach involves leveraging direct integrations into your source code management tools like GitHub, GitLab, Bitbucket, or Azure DevOps. By leveraging a native integration, a pipelineless security solution scans every event, starting from the moment of ‘git push’. The timing of the scan occurs before code is merged into the production branch and there is explicitly no configuration within the pipeline. Underpinning the integration into your source code management tool – and, in fact, made possibly because of this integration – are three core workflows:
Developers are great at writing code. They are often less adept at the git commands used for things like removing a hardcoded secret from commit history or automating CODEOWNERS file management. A pipelineless approach incorporates deep git command automation in order to keep developers focused on their code.
Developers have no deep love for ticketing systems. Chat, however, has evolved greatly over the past decade to the point where chat bots can do everything for software engineering teams, which is how mature platform engineering teams stream value. Pipelineless security leverages rich chat integrations to push critical fixes (and even 1-click mitigations) directly into the developer’s chat window to ensure the developer is empowered to make a positive impact without departing from the workflows and tools they use and love.
A core intention of the pipelineless approach is to eliminate as much risk as possible before the code is merged into your production environment. However, developers and security teams know that often a developer will prioritize a feature release before they address a non-critical security issue. In this instance you need powerful integrations into your ticketing system, such as Jira. But in order to avoid a bloated security backlog, issues should be auto closed when they are resolved in code and merged into the production environment (not beforehand!), which is “knowable” because, again, you’ve integrated pipelineless security directly into your source code management tools.
Full coverage, always. Pipelineless security gives you 100% coverage of your development ecosystem from day 1. Because you’re leveraging an integrated app for your source code, every repository is covered, regardless of its business importance. Policies, rather than pipeline configurations, are used to manipulate alerts, notifications, and auto mitigation actions based on severity, business importance of an asset, and more to keep development velocity undisturbed while making critical fixes as easy as possible.
Blameless & shameless developer feedback. The longstanding tension between developers and security is well documented. A major positive impact of pipelineless security is that it changes the interaction model from one where vulnerable code is not broadcast in a public ticket for the whole team to see, but rather provided directly to the developer in chat with deep context and optionality.
For example, if a developer pushes code that contains a vulnerable third party package, Arnica’s pipelineless security approach will send a Slack or Microsoft Teams message to the developer containing: 1) multiple mitigation paths and 2) the impact and effort of those paths (for example, in SCA, maybe a minor patch fixes only 80% of critical vulnerabilities; maybe the major change actually introduces a new critical). The developer is armed in real time with all the information needed to decide if they are going to fix the vulnerability now and which path they will take to do so.
Make fixes easy. One key reason for the design of pipelineless security, integrated into source code, is that rich and thoughtful integrations allow Application Security tools to make fixes as easy as possible without taking the developer out of their workflow. In some cases, we can even automate certain fixes.
For example, in Arnica, if a developer commits a hardcoded secret and it is identified as a true positive based on Arnica’s secret validators, rather than create a ticket to be picked up at some later time, Arnica provides an option to the developer to ‘fix it for me’. What that means under the hood is that Arnica clones the commit with the secret removed, and equally critically, automates the git commands to remove the secret from git history.
Automated risk lifecycle management. Ultimately, not every vulnerability will get handled in real time, as much as the AppSec team might daydream about that happening. When low and medium risks are deemed acceptable in the short term, tickets can be automatically created. But that is the easy part. Where the pipelineless approach stands apart is – since security is integrated directly into the source code management solution – by scanning all code not just vulnerable code. That means that when a package is updated to eliminate vulnerabilities or a low reputation package is replaced, no need to go dig up the ticket, your pipelineless security solution should have awareness of that update and can close the issue for you.
Try Arnica’s pipelineless application security platform today!
