Cloud Secret Manager Integration

version 2.18

Overview

Organizations using cloud platforms need a way to pull secrets directly from their cloud provider’s secret management service at runtime, avoiding credential duplication and leveraging existing rotation policies. This feature adds native integrations with AWS Secrets Manager and Azure Key Vault as Enterprise-only capabilities.

Motivation

Enterprise teams already manage secrets in AWS Secrets Manager or Azure Key Vault with established rotation policies, access controls, and audit trails. Duplicating these credentials into Semaphore’s database creates a second source of truth that can drift out of sync.
Credential rotation in the cloud provider does not automatically propagate to Semaphore — someone must manually update every affected key store entry after each rotation.
Compliance and security policies in regulated industries often require that secrets never leave the cloud provider’s boundary or be stored in third-party databases.
Semaphore’s built-in key store lacks the access controls, audit granularity, and rotation automation that cloud secret managers provide out of the box.

Featues

1. AWS Secrets Manager Integration

Goal: Pull secrets at runtime from AWS Secrets Manager, allowing teams to leverage their existing AWS secrets infrastructure without duplicating credentials in Semaphore’s database.

Requirements:

New external secret backend type: aws_secrets_manager.
Authentication methods:
- IAM role (recommended for EC2/ECS/EKS deployments) — no credentials stored in Semaphore.
- Access key + secret key — stored in Semaphore’s key store (encrypted).
Secrets are referenced by name.
Support for JSON-structured secrets with a field selector to extract individual values (e.g., {"username": "admin", "password": "..."} → extract password).
Automatic secret rotation support — when AWS rotates a secret, Semaphore picks up the new value on the next task run.

Related issues: #2248

2. Azure Key Vault Integration

Goal: Pull secrets at runtime from Azure Key Vault, enabling organizations using Azure to centrally manage and rotate credentials without storing them in Semaphore.

Requirements:

New external secret backend type: azure_key_vault.
Authentication methods:
- Managed identity (recommended for Azure VM/AKS deployments) — no credentials stored in Semaphore.
- Service principal with client secret.
- Service principal with certificate.
Secrets are referenced by vault URL + secret name.
Support for retrieving secrets from the vault.
Automatic secret rotation support — when Azure rotates a secret, Semaphore picks up the new value on the next task run.

Related issues: #2248, #3170

3. Secret Caching

Goal: Reduce API calls to cloud providers by caching resolved secrets in memory.

Requirements:

Optional short-lived in-memory cache (configurable TTL, default 5 minutes) to reduce API calls to the cloud provider.
Cache is in-memory only and is invalidated on Semaphore restart.
Cache is keyed by backend config ID + secret reference, so the same secret used by multiple templates is fetched only once per TTL window.
Cache can be disabled per backend configuration (set TTL to 0) for secrets that must always be fetched fresh (e.g., during active rotation).
Configuration: cache_ttl field on the backend configuration (e.g., "cache_ttl": "5m").

Configuration

New configuration options:

Key	Env Var	Default	Description
`secret_cache_ttl`	`SEMAPHORE_SECRET_CACHE_TTL`	`5m`	Default in-memory cache TTL for resolved external secrets
`secret_resolve_timeout`	`SEMAPHORE_SECRET_RESOLVE_TIMEOUT`	`10s`	Maximum time to wait for a cloud provider API response when resolving a secret