CI/CDSCApipeline performance by Derek Voss

SCA in High-Velocity Pipelines: Security That Doesn't Slow You Down

Engineering orgs shipping 50+ deploys per day need security tooling that matches their velocity. Batch-scan approaches built for weekly release cycles don't cut it.

SCA in High-Velocity Pipelines: Security That Doesn't Slow You Down

Software composition analysis was designed for a different deployment cadence. The classic SCA workflow — weekly batch scan, ticket created in Jira, prioritized in next quarter's sprint — made sense when teams shipped every two weeks. It does not make sense when your engineering org is opening 40 PRs a day and deploying to production multiple times before lunch.

The mismatch isn't just procedural. High-velocity deployment pipelines require a different architecture for security tooling, different latency expectations, and a fundamentally different theory of where in the workflow security checks create value.

Why batch-scan SCA breaks at high velocity

Batch scanning operates on a schedule — daily or weekly — against the current state of your main branch or deployed environment. The fundamental problem is temporal: by the time the scan runs and the ticket is created, the code it flagged has often been superseded by three more PRs. The dependency that was introduced in PR #847 was already removed or updated in PR #851 before the scan output was even read.

Batch scans also create batch workloads. If your scanner runs weekly and surfaces 25 alerts, that's 25 alerts landing on someone's queue simultaneously, creating a triage event that competes with everything else happening that day. Some of those alerts are stale (the relevant code already changed). Some are duplicates of last week's ignored alerts. A few are genuinely new and important. But the batching structure means all 25 arrive with equal urgency and no context about which PR introduced each issue.

The "owning team" problem gets worse at velocity too. A weekly scan can tell you a vulnerable package is in your lockfile; it can't easily tell you which PR introduced it, who authored that PR, or what the original intent of the change was. That context is critical for efficient remediation.

PR-level scanning: the architecture shift

PR-level SCA inverts the batch model entirely. Instead of "scan the world on a schedule," the trigger is "a PR changed something in the dependency manifest." The scan scope is the delta — what changed in this PR's lockfile or dependency spec — not the full lockfile on every scan.

This scoping is essential for pipeline performance. Scanning only the CVEs relevant to the lockfile diff means analysis time is proportional to the size of the dependency change, not the total size of the lockfile. A PR that updates one package version triggers analysis of the CVEs affecting that package version, not a re-scan of all 900 packages in the lockfile.

In practical terms: a typical PR that updates 2-3 direct dependencies (and their transitive deps) should complete its security check in 10-30 seconds for most codebases. That fits within the acceptable latency window for a CI check — developers expect checks to complete before they've finished their next task. Batch scans optimized for weekly schedules routinely take minutes; that's incompatible with PR feedback loops.

The result-feedback loop for developer workflow

At 50+ deploys per day, the window between a developer merging a PR and that code reaching production is measured in hours, not days. Security findings that surface after merge have already lost the optimal remediation window — the developer's context is stale, they've moved to other work, and fixing the issue requires context-switching back to a PR they considered done.

PR-level scanning keeps the finding in the developer's workflow context. When the security check runs as part of the PR status check suite — alongside tests, linting, and type checks — the developer is already in the code. They see the reachability finding, understand the affected code (because they just wrote it), and can either update the package version immediately or make an informed decision to defer with context.

Consider a backend engineering team with a 90-deploy-per-day pipeline. Their previous security tooling ran a nightly scan that generated a Jira backlog. The backlog accumulated faster than it was resolved, reaching 200+ open tickets after three months. Most of the backlog items had lost meaningful context — nobody could trace them back to a specific PR, the original author, or the original intent of the change. Remediation rate was effectively zero.

After switching to PR-level scanning with inline PR comments, their security finding resolution rate changed dramatically: findings surfaced at PR time were resolved in the same PR 80% of the time, because the developer could see the issue, understood the context, and the fix (usually a version bump to a non-vulnerable release) was immediately at hand.

Latency requirements and where to absorb them

Not all latency is equal in a CI pipeline. There's a meaningful difference between:

  • Required checks — checks that block merge. These need to complete fast (under 60 seconds ideally, under 3 minutes as an outer bound). Developers wait for required checks; slow required checks get bypassed or ignored.
  • Advisory checks — checks that post a result but don't block merge. These can run on a slightly longer window and post a comment or status update when complete.

For reachability-based SCA, the right model depends on what the tool is checking. A PR that introduces a new direct dependency with a CRITICAL reachable CVE should absolutely block merge — that's a hard signal. A PR that updates a transitive dependency and the reachability status is ambiguous (dynamic import path, uncertain call chain) might be better as an advisory comment rather than a merge block.

The tooling needs to support this nuance. A security check that blocks every PR with any CVE finding trains developers to bypass the check. A security check that only blocks on REACHABLE + CRITICAL (or configurable severity thresholds) trains developers to respect it, because it's blocking on evidence, not noise.

SBOM generation at PR cadence

A secondary benefit of PR-level scanning at high velocity: continuous SBOM (Software Bill of Materials) generation. Rather than generating an SBOM against a point-in-time snapshot, your SBOM reflects the state of your dependency graph after every merge. For organizations subject to executive order EO 14028 requirements or pursuing SOC 2 Type II with security controls around third-party software tracking, a continuously updated SBOM generated from a PR-triggered pipeline is a much more defensible artifact than a weekly batch output.

We're not saying SBOM compliance is the primary reason to shift to PR-level scanning — developer workflow and false positive reduction are — but it's a meaningful secondary benefit that often accelerates organizational buy-in for the tooling investment.

Pipeline performance: the non-negotiable constraint

Engineering orgs at high velocity have one non-negotiable requirement for any CI tooling: don't be the bottleneck. A security check that adds 5 minutes to every PR at 50 PRs per day adds 4+ hours of accumulated pipeline wait time daily. That's a real productivity cost, and engineering teams will route around it.

The architectural answer is bounded analysis scope (delta-only, not full-lockfile), parallelization of CVE lookup and call graph tracing, and a caching layer for call graph analysis results across PRs that touch the same packages. The incremental reachability problem — "given that we computed call graph reachability for this package version yesterday, what changes in today's PR affect that analysis?" — is a tractable optimization that keeps PR-level scan times in the acceptable range.

Security tooling that respects pipeline performance isn't making a security concession — it's making a practical bet that tools developers actually run catch more vulnerabilities than tools they route around.

PR-Level Security Gates That Don't Friction Your Developers A Practical Primer on Call Graph Analysis for Application Se...