Scaling workers

Size the worker pools of a self-hosted sfp server — worker count, slots per worker, and the standard vs long-running pools in docker compose.

The sfp server runs work — builds, releases, scratch-org and sandbox provisioning, PR validation — on background workers, not on the server process. Each unit of work is a task the server hands to a worker. How much work the server can run at once is set by how many workers are running and how many tasks each worker takes concurrently.

Two worker pools run side by side:

Pool	Handles	Default count	Slots per worker
Standard	Short and medium tasks — builds, deployments, validations	4	2
Long-running	Heavy tasks — scratch-org and sandbox provisioning	2	1

The two pools are separate so that a burst of slow provisioning tasks cannot starve the fast build and deployment tasks, and vice versa. Long-running workers take one task at a time because those tasks are heavy and long; standard workers take two because their tasks are shorter.

Count times slots

A worker's slots are how many tasks it runs concurrently. A pool's total concurrency is its worker count multiplied by the slots per worker.

standard concurrency     = worker count      × 2 slots
long-running concurrency = long-running count × 1 slot

With the defaults, the server runs up to 4 × 2 = 8 standard tasks and 2 × 1 = 2 long-running tasks at the same time.

Adding workers and adding slots both raise concurrency, but they trade off differently:

• More workers spread load across more processes. A worker crash takes down only its own in-flight tasks, and each worker is a separate container the host scheduler can place independently. This is the safer axis to scale.
• More slots pack more concurrent tasks into each worker process. It uses less memory overhead per task, but a worker crash takes down more in-flight tasks at once, and all slots in a worker share that process's resources.

Setting pool sizes

Rescale the standard pool with sfp server scale

sfp server scale is the supported way to change the standard pool's worker count. It applies the new count to the running services immediately. To make the count survive the next start or update, also set HATCHET_WORKER_COUNT=<n> in the tenant's .env — the compose file reads the replica count from that variable.

# Scale the standard pool to 6 workers
sfp server scale --tenant <tenant> --workers 6

To see how many workers are actually running, use sfp server status — it lists each worker service and its state.

--workers accepts 1–10. Add --ssh-connection user@host[:port] and --identity-file <key> to run it against a remote server from your workstation.

Pool defaults in compose

The declared replica counts and the slots per worker come from the compose file. The worker counts default from environment variables; the slots are fixed per pool.

Setting	Where	Default
Standard worker count	HATCHET_WORKER_COUNT (compose)	4
Long-running worker count	HATCHET_LONGRUNNING_WORKER_COUNT (compose)	2
Standard slots per worker	fixed in compose	2
Long-running slots per worker	fixed in compose	1

sfp server scale rescales the standard pool. To change the long-running pool count, set HATCHET_LONGRUNNING_WORKER_COUNT for the tenant and restart the server so compose recreates that pool at the new count.

Sizing guidance

Worker capacity is bounded by the host's CPU and memory, and by the Salesforce work each task does. Start from the defaults and adjust against what you observe.

• Match concurrency to the host. Each standard slot can run a build or deployment, which is CPU- and memory-heavy while active. Do not set total standard concurrency (count × 2) higher than the host can run without contention — oversubscribing slows every task rather than adding throughput. Leave headroom for the server, the Hatchet engine, and the databases, which run on the same host.
• Scale the pool that is the bottleneck. If builds and deployments queue while the host still has capacity, add standard workers with sfp server scale. If sandbox or scratch-org provisioning is the backlog, raise the long-running pool.
• Prefer adding workers over adding slots. More workers give better fault isolation and let the host schedule each independently. Slots are fixed per pool by design; scale the pool by count.
• Use the queue depth to decide. The server's /sfp/api health endpoint reports activeWorkflows: { running, queued, total } across both pools. A persistently high queued count with spare host capacity is the signal to add workers; a high count with the host already saturated means the host is the limit, not the worker count.

Related pages

Monitoring for sfp serverUpdating sfp server