Skip to main content
How a Vault License Change Exposed a Secrets Management CrisisIncident
4 min readFor Security Engineers

How a Vault License Change Exposed a Secrets Management Crisis

What Happened

In August 2023, HashiCorp transitioned Vault from the Mozilla Public License to the Business Source License (B.S.L.). This change didn't cause a data breach, but it revealed a critical issue: many organizations found themselves without a backup plan for secrets management. The new license restricts commercial use of the open-source version, pushing teams to either purchase enterprise licenses or quickly find alternatives.

The real problem wasn't a data leak but the sudden awareness that essential security infrastructure was vulnerable to a single vendor's decision. For teams managing machine learning (ML) workloads that require extensive data access, this created an immediate crisis in managing secrets exposure.

Timeline

August 10, 2023: HashiCorp announces the B.S.L. for Vault and other products, effective immediately for new releases.

August-September 2023: Engineering teams conduct impact assessments. Organizations realize they are running Vault instances for ML pipelines, CI/CD workflows, and production services without a migration plan.

October 2023-Present: Teams evaluate alternatives like Infisical, Doppler, and AWS Secrets Manager while maintaining existing Vault deployments. Some freeze Vault at pre-B.S.L. versions, creating technical debt and security risks.

Ongoing: ML systems face increasing complexity as they require access to more services and data sources, multiplying the secrets that need migration or re-architecting.

Which Controls Failed or Were Missing

The licensing change didn't break controls; it exposed their absence:

Vendor lock-in without contingency planning: Teams treated Vault as infrastructure, not a vendor relationship needing an exit strategy. No procedures existed for migrating secrets to alternative systems.

Secrets sprawl in ML pipelines: ML systems need broad access to data and services. Teams found secrets scattered across Jupyter notebooks, training scripts, and CI/CD pipelines without a central inventory. They couldn't answer basic questions about the number of secrets, their locations, or their activity status.

Missing secrets lifecycle management: Organizations lacked processes for rotating, expiring, or auditing secrets independently of Vault. The tool became the process, rather than implementing the process through the tool.

No separation of secrets management from secrets storage: Teams equated "using Vault" with "managing secrets." They had storage, not management—no policies for secret creation, approval workflows, or usage monitoring.

What the Relevant Standards Require

PCI DSS v4.0.1 Requirement 8.3.2 mandates protecting authentication credentials during transmission and storage, with specific controls for cryptographic key management. It requires processes for protecting secrets that survive vendor changes.

ISO/IEC 27001:2022 Control 8.3 requires restricting access to information according to established policies. Access controls should be policy-driven, not tool-dependent. If your access control can't survive a vendor migration, you have vendor dependency, not control.

NIST 800-53 Rev 5 Control IA-5 requires managing system authenticators with procedures for distribution, loss/compromise, and revocation, independent of technology.

SOC 2 Type II CC6.1 requires documented processes for managing credentials and secrets. Auditors need to see policies, approval workflows, and monitoring—not just "we deployed Vault."

None of these standards accept "our vendor handles it" as compliance evidence. They require documented processes that can be implemented through various technical solutions.

Lessons and Action Items for Your Team

Document your secrets inventory now: List every secret your systems use—API keys, database credentials, service account tokens, encryption keys. For ML systems, include model registry credentials, data lake access tokens, and inference endpoint keys. Use automated tools but verify manually. You can't migrate what you can't enumerate.

Implement secrets lifecycle policies independent of tooling: Define who can create secrets, their validity, rotation requirements, and approval workflows. Write these as policies first, then configure your tool to enforce them. When switching tools, you'll migrate policy enforcement, not reinvent policy.

Build a vendor exit plan for critical security infrastructure: For any security tool that could cause an operational crisis if unavailable, document alternative solutions, migration complexity, data export procedures, and a timeline to switch. Review annually. Treat security infrastructure like cloud providers—assume you might need to move.

Separate ML secrets from application secrets: ML workloads access different resources than traditional applications. Create separate secret stores or namespaces for ML infrastructure. This limits the blast radius and simplifies compliance when ML systems access sensitive data.

Test your secrets management under migration pressure: Run a tabletop exercise where your current secrets management solution becomes unavailable in 90 days. Can your team identify all secrets, prioritize migration, and execute without disruption? If not, you have process gaps that will surface during actual vendor changes.

Audit secrets sprawl in CI/CD pipelines: ML teams often embed credentials in scripts and pipelines. Scan repositories for hardcoded secrets. Implement pre-commit hooks to block credential commits. For ML-specific workflows, use short-lived credentials tied to job execution rather than long-lived tokens.

The HashiCorp licensing change is a wake-up call. Use it to build secrets management that withstands vendor decisions, mergers, and infrastructure shifts your organization doesn't anticipate.

Topics:Incident

You Might Also Like