Scaling workers

Size the worker pools of a self-hosted sfp server — worker count, slots per worker, and the standard, long-running and sandbox pool workers in docker compose.

The sfp server runs work — builds, releases, scratch-org and sandbox provisioning, PR validation — on background workers, not on the server process. Each unit of work is a task the server hands to a worker. How much work the server can run at once is set by how many workers are running and how many tasks each worker takes concurrently.

Three worker pools run side by side:

PoolHandlesDefault countSlots per worker
StandardShort and medium tasks — builds, deployments, validations42
Long-runningHeavy tasks — scratch-org and sandbox provisioning21
Sandbox poolThe per-sandbox deployments of a sandbox pool sync12

The pools are separate so that a burst of slow provisioning tasks cannot starve the fast build and deployment tasks, and vice versa. Long-running workers take one task at a time because those tasks are heavy and long; standard workers take two because their tasks are shorter.

A sandbox pool sync deploys a branch to every available sandbox in a pool, one task per sandbox. Those tasks run only on the sandbox pool workers, so a scheduled sync of a large pool does not take the standard workers' slots, and builds, releases and validations start while it runs. The pool's Sandboxes at a time setting (batchSize) sets how many sandbox tasks a sync queues at once. How many of them run at once is capped by the sandbox pool's concurrency, across all pools on the server; the rest wait for a free slot.

Count times slots

A worker's slots are how many tasks it runs concurrently. A pool's total concurrency is its worker count multiplied by the slots per worker.

standard concurrency     = worker count      × 2 slots
long-running concurrency = long-running count × 1 slot
sandbox pool concurrency = pool worker count  × pool slots (2 by default)

With the defaults, the server runs up to 4 × 2 = 8 standard tasks, 2 × 1 = 2 long-running tasks and 1 × 2 = 2 sandbox syncs at the same time.

Adding workers and adding slots both raise concurrency, but they trade off differently:

  • More workers spread load across more processes. A worker crash takes down only its own in-flight tasks, and each worker is a separate container the host scheduler can place independently. This is the safer axis to scale.
  • More slots pack more concurrent tasks into each worker process. It uses less memory overhead per task, but a worker crash takes down more in-flight tasks at once, and all slots in a worker share that process's resources.

Setting pool sizes

Rescale with sfp server scale

sfp server scale is the supported way to change a pool's worker count. It applies the new count to the running services immediately and writes it to the tenant's .env (HATCHET_WORKER_COUNT, HATCHET_LONGRUNNING_WORKER_COUNT, HATCHET_POOL_WORKER_COUNT), so the count survives the next start or update.

FlagPoolRange
--workersStandard1–10
--longrunning-workersLong-running1–10
--pool-workersSandbox pool1–10
# Scale the standard pool to 6 workers
sfp server scale --tenant <tenant> --workers 6

# Run up to 4 sandbox syncs at once (2 workers × 2 slots)
sfp server scale --tenant <tenant> --pool-workers 2

--workers without --longrunning-workers also sets the long-running pool to half the standard count, rounded up. A pool without a flag keeps the count it is running. When a scale removes workers, sfp server scale first waits for in-flight work to finish; --force removes them without waiting.

Without a count flag, sfp server scale prints each pool's configured and running counts. sfp server status lists each worker service and its state.

Add --ssh-connection user@host[:port] and --identity-file <key> to run it against a remote server from your workstation.

Pool defaults in compose

The declared replica counts and the slots per worker come from the compose file. The worker counts default from environment variables; the slots are fixed per pool.

SettingWhereDefault
Standard worker countHATCHET_WORKER_COUNT (compose)4
Long-running worker countHATCHET_LONGRUNNING_WORKER_COUNT (compose)2
Sandbox pool worker countHATCHET_POOL_WORKER_COUNT (compose)1
Standard slots per workerfixed in compose2
Long-running slots per workerfixed in compose1
Sandbox pool slots per workerHATCHET_POOL_WORKER_SLOTS (compose)2

Keep at least one sandbox pool worker on a server with sandbox pools. No other pool runs sandbox syncs, so without one they stay queued. --pool-workers does not accept 0.

The sandbox pool worker container is created by sfp server update (or sfp server start) on a release that defines it.

Sizing guidance

Worker capacity is bounded by the host's CPU and memory, and by the Salesforce work each task does. Start from the defaults and adjust against what you observe.

  • Match concurrency to the host. Each standard slot can run a build or deployment, which is CPU- and memory-heavy while active. Do not set total standard concurrency (count × 2) higher than the host can run without contention — oversubscribing slows every task rather than adding throughput. Leave headroom for the server, the Hatchet engine, and the databases, which run on the same host.
  • Scale the pool that is the bottleneck. If builds and deployments queue while the host still has capacity, add standard workers with sfp server scale. If sandbox or scratch-org provisioning is the backlog, raise the long-running pool. If sandbox pool syncs take too long, raise the sandbox pool with --pool-workers; each sandbox sync deploys to its own sandbox, so more of them at once add load on Salesforce as well as on the host.
  • Prefer adding workers over adding slots. More workers give better fault isolation and let the host schedule each independently. Slots are fixed per pool by design; scale the pool by count.
  • Use the queue depth to decide. The server's /sfp/api health endpoint reports activeWorkflows: { running, queued, total } across all pools. A persistently high queued count with spare host capacity is the signal to add workers; a high count with the host already saturated means the host is the limit, not the worker count.

On this page