What's new in Semaphore UI 2.19

Semaphore UI 2.19 is the largest release of the 2.x line. It introduces Workflows with a graphical editor, lets runners execute tasks in Docker containers and Kubernetes pods, teaches Semaphore to issue short‑lived JWT identity tokens to running tasks, adds encryption key rotation, fixes runner reliability for good, and ships a long list of security hardening changes.

The line covers releases v2.19.0 through v2.19.14. Patch releases are listed at the end.

Highlights #

• Workflows (Beta) — chain task templates into a pipeline with approval gates, drawn in a graphical editor and watched live on the same canvas.
• Docker and Kubernetes executors — runners can execute every task in a fresh container or pod (Pro / Enterprise).
• JWT identity tokens for tasks — keyless authentication from playbooks to Vault, AWS, GCP, Azure and anything else that accepts OIDC tokens.
• Encryption key rotation — a labelled keyring with hot reload and a vaults check command.
• Survey variables as environment variables, plus int, text and restyled enum types.
• Real server‑side pagination of task history — projects with millions of tasks stay fast.
• Runner reliability — online/offline status, one‑time hashed registration tokens, and automatic recovery of tasks stuck on a dead runner.
• Selective debug logging with SEMAPHORE_DEBUG_FILTER.
• Security hardening: current‑password check, CSRF origin validation, secure cookies, stricter input validation across the API.
• BoltDB removed — SQLite, MySQL and PostgreSQL only.

---

Workflows (Beta) #

A workflow is a graph of task templates that runs as one unit. Each node is either a task (runs a template), an approval (pauses the run until a user approves or rejects), or a note (free‑form annotation that never executes). Edges carry a condition: on success, on failure or always.

Workflows appear as a new Workflows item in the project sidebar, marked with a Beta chip. They are available in the Pro edition; during the beta they are enabled regardless of plan.

Graphical editor #

The editor is a full‑page canvas built on Drawflow:

• drag nodes from the palette, connect them by dragging from a node handle;
• click an edge to change its condition; edges are colour‑coded and a legend sits in the corner;
• click a node to edit its properties in the side panel: template, convergence mode (all parents / any parent), approval timeout and message, note text;
• self‑edges and cycles are rejected as you draw them, and a Problems panel mirrors the server‑side validation so a broken graph cannot be saved;
• node positions are persisted; workflows created through the API without positions get an automatic topological layout;
• zoom with the toolbar or Ctrl + mouse wheel, drag the canvas to pan, collapse the palette to gain space;
• Start version seeds run versioning (1.4.0, 1.4.1, …); the version is propagated to every task the run launches.

Live run view #

The run view reuses the same canvas. Each node shows its task status, the active node is highlighted, and pending approvals render Approve / Reject buttons directly on the canvas. A Stop button force‑stops every task of the run, rejects pending approvals and marks the run stopped.

Run statuses are running, approval, success, failed and stopped. Workflow progression is driven by the server: when any workflow task finishes, the next nodes are scheduled, so a template without autorun children no longer stalls the run.

Per‑node task parameters #

Every task node can override the parameters it passes to its template (variables, inventory, branch, arguments, version, message), the same way a manual run can.

API #

GET/POST   /api/project/{id}/workflows
GET/PUT/DELETE /api/project/{id}/workflows/{workflow_id}
POST       /api/project/{id}/workflows/{workflow_id}/run
GET        /api/project/{id}/workflows/{workflow_id}/runs
GET        /api/project/{id}/workflows/{workflow_id}/runs/{run_id}
POST       /api/project/{id}/workflows/{workflow_id}/runs/{run_id}/stop
GET        /api/project/{id}/workflows/{workflow_id}/runs/{run_id}/artifacts
GET        /api/project/{id}/workflows/{workflow_id}/runs/{run_id}/approvals
POST       /api/project/{id}/workflows/{workflow_id}/runs/{run_id}/approvals/{node_id}

Known limitations of the beta: no minimap, undo/redo or multi‑select; workflow artifacts (set_stats values) do not yet flow between tasks executed on remote runners.

---

Docker and Kubernetes executors (Pro / Enterprise) #

Runners can execute each task in an isolated environment instead of directly on the runner host. The executor is selected once per runner process with runner.executor.type (local, docker, k8s).

Docker (runner.executor.docker, Pro):

Option	Env var	Default
host	SEMAPHORE_RUNNER_DOCKER_HOST	local socket
tls_verify, cert_path	SEMAPHORE_RUNNER_DOCKER_TLS_VERIFY, …_CERT_PATH
image	SEMAPHORE_RUNNER_DOCKER_IMAGE	semaphoreui/job:latest
helper_image	SEMAPHORE_RUNNER_DOCKER_HELPER_IMAGE	semaphoreui/helper:latest
network	SEMAPHORE_RUNNER_DOCKER_NETWORK	bridge
pull_policy	SEMAPHORE_RUNNER_DOCKER_PULL_POLICY	if-not-present
cpu_limit, memory_limit	SEMAPHORE_RUNNER_DOCKER_CPU_LIMIT, …_MEMORY_LIMIT
privileged	SEMAPHORE_RUNNER_DOCKER_PRIVILEGED	false
poll_interval_seconds, cleanup_grace_seconds	…_POLL_INTERVAL_SECONDS, …_CLEANUP_GRACE_SECONDS	2, 30

Kubernetes (runner.executor.k8s, Enterprise):

Option	Env var	Default
kubeconfig	SEMAPHORE_RUNNER_K8S_KUBECONFIG	in‑cluster
namespace	SEMAPHORE_RUNNER_K8S_NAMESPACE	semaphore
image	SEMAPHORE_RUNNER_K8S_IMAGE	semaphoreui/job:latest
helper_image	SEMAPHORE_RUNNER_K8S_HELPER_IMAGE	semaphoreui/helper:latest
service_account	SEMAPHORE_RUNNER_K8S_SERVICE_ACCOUNT	default
pull_secrets	SEMAPHORE_RUNNER_K8S_PULL_SECRETS
poll_interval_seconds, cleanup_grace_seconds	…_POLL_INTERVAL_SECONDS, …_CLEANUP_GRACE_SECONDS	3, 30

Two new images are published by CI: semaphoreui/job (Ansible, Terraform, OpenTofu, Terragrunt, paramiko) and semaphoreui/helper. A template can override the image for its own tasks with the new Executor image field in the template form (shown with an Upgrade to PRO badge when the executor feature is not licensed).

The runner connection also gained runner.connection.server_ca_cert_file and runner.connection.skip_tls_verify.

---

JWT identity tokens for tasks #

Semaphore can act as an OIDC‑style identity provider for running tasks, so a playbook can authenticate to Vault, cloud STS endpoints or internal services without long‑lived credentials.

• Enable it per template with Issue JWT to task runner; set one or more audiences and a TTL (capped by jwt.max_ttl).
• The task receives the token in the SEMAPHORE_JWT environment variable.
• Tokens are ES256 (ECDSA P‑256) signed and carry only IDs: task_id, project_id, template_id, user_id plus the standard iss, sub, aud, exp, nbf, iat, jti claims.
• Public keys are published at GET /.well-known/jwks.json.

Server configuration:

"jwt": {
  "enabled": true,
  "issuer": "https://semaphore.example.com",
  "default_ttl": "1h",
  "max_ttl": "24h"
}

Environment variables: SEMAPHORE_JWT_ENABLED, SEMAPHORE_JWT_ISSUER, SEMAPHORE_JWT_DEFAULT_TTL, SEMAPHORE_JWT_MAX_TTL.

---

Survey variables #

Deliver a variable as an environment variable #

A survey variable now has a Pass variable as setting: the app‑specific CLI way (--extra-vars for Ansible, -var for Terraform/OpenTofu, a CLI argument for shell scripts) or a process environment variable. The variable name is used verbatim, so a Terraform user simply names it TF_VAR_region. Environment variables do not show up in process listings, which makes this the safer choice for secrets.

New types #

• int — numeric input with validation;
• text — multi‑line text;
• enum — restyled editor with name/value pairs and a default.

The task dialog renders each type accordingly:

Stored in the existing survey_vars JSON — no migration required.

---

Templates and tasks #

• Dynamic playbook picker. The Path to playbook file field lists the playbooks found in the repository (GET /api/project/{id}/repositories/{repository_id}/playbooks). The list follows the selected branch and falls back to free text when the repository cannot be read.

• Skip Ansible Galaxy install — per‑template option, optionally overridable per task, to skip installing roles and collections from requirements.yml.

• Typed variables in variable groups, including numbers:

  

• Task history pagination. The History page used to fetch the newest 200 tasks and page through them client‑side, so anything older was unreachable. The backend now returns one page at a time using a keyset cursor (?count=20&before=<task_id>, X-Has-Next response header) with no COUNT(*) and no OFFSET. The footer offers Rows per page and previous/next controls. The same pagination applies to the per‑template task list and the dashboard.

  

• Task lists reload at most once every 5 seconds; several redundant requests were removed.

• Schedules are validated with the server‑side cron parser, so client and server no longer disagree.

• Branch override in a task is only accepted when the template allows it.

• Git operations are serialized per repository directory. Templates with Allow parallel tasks share one working copy, and concurrent git pull / git checkout could corrupt it. Update and checkout now form a single critical section, for both local and runner execution, including inventory repositories.

---

Runners #

• Online / offline status on the Runners page, derived from heartbeat liveness.

• One‑time registration tokens. A runner can be created in the UI first and registered later with a smrs_… token that is shown once, stored only as a SHA‑256 hash and expires after one hour. The dialog shows ready‑to‑copy commands for env vars, config file and Docker. Regenerating the token resets an already‑registered runner so it can be re‑registered.

  

  SEMAPHORE_WEB_ROOT=https://semaphore.example.com \
  SEMAPHORE_RUNNER_REGISTRATION_TOKEN=smrs_… \
  semaphore runner register --config ./config.runner.json
  
  semaphore runner start --config ./config.runner.json
  

• Hung‑task recovery. Runners send their process start time (X-Runner-Started-At). A runner that stops polling is marked offline after runners.offline_timeout_sec (120 s): it receives no new tasks and its starting tasks are reassigned. After runners.task_fail_timeout_sec (420 s) its running tasks are failed with a clear message. A runner that restarted and lost its in‑memory job pool is detected immediately. Reconciliation runs every runners.reconcile_interval_sec (30 s).

• Tasks reassigned away from a runner are terminated on the old runner.

• The stale‑runner fallback is gone: when every runner is offline, tasks wait in the queue instead of being dispatched to a runner that has not polled for up to 30 minutes.

• Per‑runner RSA encryption keys removed. Runner‑server traffic relies on TLS; this deletes the key exchange step from registration and setup.

• Fixed a TCP connection leak in the runner client; invalid registration tokens return 400.

---

Secrets and encryption #

• Encryption key rotation. The new encryption block describes a labelled keyring: inline keys (value or file), or a keys_folder where every file is a key named by its filename, plus active pointers for the secret key and the option key. Ciphertext now carries a key ID, so keys can be rotated without a big‑bang re‑encrypt. encryption.keys_file with keys_poll_interval (default 15s) hot‑reloads the keyring. New CLI: semaphore vaults check; semaphore vaults rekey was rewritten around the keyring. The legacy flat access_key_encryption still works.
• option_encryption — a separate key for options stored in the database.
• OpenBao secret storage type (routed through the Vault provider) with its own icon.
• AWS Secrets Manager without static credentials — a Use IAM role checkbox.
• TLS‑skip‑verify option for Vault/OpenBao storages.
• Synchronized and read‑only secret fields are no longer wiped on update.

---

Observability #

• Namespaced debug logging. --debug-filter / SEMAPHORE_DEBUG_FILTER selects which subsystems emit debug output, Node.js debug‑style: runner, runner,task_pool, task_*, *, *,-db. Available namespaces: runner, task_pool, task_runner, task_logger, git, terraform, session, ldap, schedule, db, ha. The filter only applies when the log level is DEBUG; it never raises the level. Syslog hooks honour the same filter.
• Many new contextual debug statements across runners, tasks and auth; the selected workspace is printed at startup.

---

Security #

• Changing a password now requires the current password (CWE‑620).
• Origin / Referer validation on state‑changing requests (CSRF hardening).
• Session cookies are marked Secure when served over HTTPS.
• Runner registration tokens are stored hashed and expire; per‑runner encryption keys were removed.
• Custom role creation checks the caller’s permissions (privilege‑escalation fix).
• Git URL validation; --end-of-options is passed to git so a crafted ref cannot be read as a flag; commit hashes are format‑checked; branches are validated before repository browsing; playbook paths are validated.
• Access key payloads and the template app field are validated.
• JWT claims carry IDs only — no names or e‑mails leak to external systems.
• Runner tokens are no longer written into project backups.
• The API returns after a write error instead of continuing with a partially written response.
• Security SLA published in SECURITY.md; release artifacts are signed with the [email protected] GPG key.

---

UI and localization #

• Czech translation.
• Dropdown cards for the JWT and schedule sections of the template form.
• Copy‑to‑clipboard icon is visible in light mode; running‑task spinners fixed; template form padding fixed; workflow Beta label.
• Integration variable extraction preserves JSON objects and arrays instead of stringifying them.

---

Upgrade notes #

Breaking and behavioural changes #

1. BoltDB is gone. bolt is no longer a valid dialect; the server refuses to start with “Bolt is not supported starting from version 2.19”. Migrate to SQLite, MySQL or PostgreSQL first.
2. Runner encryption keys removed. Server and runners must both be on 2.19. Make sure runner‑server traffic is TLS‑protected. Scripted runner provisioning that expected the key must be updated.
3. Task list APIs are paginated. GET /api/project/{id}/tasks/last takes count and before; limit is still accepted, but clients that relied on the newest 200 tasks in one response must page.
4. No stale‑runner fallback. With all runners offline, tasks stay queued.
5. Runner active flag removed from registration.
6. Project backups no longer contain runner tokens.
7. SQLite: v2.19.14 rebuilds the session and task tables to add proper foreign keys (fixes user deletion). Orphaned sessions are removed. Back up the database before upgrading.

New configuration #

jwt, runners, encryption, option_encryption, secrets_path, db.dialect, runner.executor.{type,docker,k8s}, runner.connection.{server_ca_cert_file,skip_tls_verify}, runner.registration_token_file, runner.token_file. All are optional; existing configs keep working. config.schema.yaml and the config reference docs were regenerated.

Database migrations #

v2.18.6 (template jwt_params), v2.18.15 (workflow tables), v2.19.2 (runner.started_at), v2.19.11 (project__workflow_node.task_params_id), v2.19.12 (project__template.executor_image), v2.19.14 (SQLite session/task rebuild). MariaDB 12.1 migration compatibility fixed.

---

Patch releases #

Version	Changes
2.19.8	Fixed SQLite migration; fixed user deletion (session/task foreign keys); case‑insensitive string comparison in DB queries
2.19.9	SEMAPHORE_RUNNER_EXECUTOR_TYPE env var; config omitempty cleanup
2.19.10	Executor config fix; Docker build option fix
2.19.11	Selected workspace printed in logs; latest tag for the helper image; DB migration tests
2.19.12	Fixed nil pointer in runner config options; error propagation fix
2.19.14	Fixed legacy secrets_path handling in config

Dependencies #

Go 1.26; go-git 5.19, go-oidc 3.19, golang.org/x/crypto 0.53, go-ldap 3.4.13, modernc.org/sqlite 1.52. Docs added as a git submodule; THIRD-PARTY-LICENSES.md regenerated.