Podman Support

Using Podman as the container engine for sfp-server deployments.

Docker is the default container engine for sfp-server. Podman is supported as an alternative through Docker Compose v2 running behind podman compose.

Select the engine with the --container-engine CLI flag or the SFP_CONTAINER_ENGINE environment variable. When set to podman, sfp-server uses the Podman adapter for runtime commands (login, volume creation, image management) and generates compose files with Podman-specific adjustments — Docker Hub image references are fully qualified, and bind-mount volumes are relabeled with :Z for SELinux compatibility.

---

[1] Prerequisites

Requirement	How to check
Podman installed	podman --version
Docker Compose v2 installed as the Compose provider	podman compose version — must report Docker Compose v2, not podman-compose
Deploy user can run Podman rootlessly (or via sudo)	podman ps without errors
SELinux graphroot label is not user_home_t (rootless RHEL/Fedora)	ls -Zd $(podman info --format '{{.Store.GraphRoot}}') — should show data_home_t or container_var_lib_t. See 1.c
Private sfp-pro images require registry credentials	See [1.a] below

podman-compose (the Python package) is not supported. sfp-server requires Docker Compose v2 as the Compose provider behind podman compose. Install the standalone docker-compose v2 binary if it is not already present.

[1.a] Registry credentials

sfp-server images are pulled from source.flxbl.io, a private registry. Podman needs credentials the same way Docker does — through the existing secrets / env / config flow described in Setting up sfp server → [1.1].

Log in before running sfp server init or sfp server start:

podman login source.flxbl.io -u <your-user> --password-stdin <<< "<your-token>"

[1.b] SELinux — bind mounts

On SELinux-enforcing hosts (RHEL, Fedora, CentOS Stream), bind mounts need the :Z label so containers can access host paths. When the container engine is set to podman, sfp-server's compose renderer adds :Z to bind-mount volumes automatically.

If you still see Permission denied errors on bind-mounted paths, relabel the host directories manually:

chcon -Rt svirt_sandbox_file_t /opt/sfp-server/tenants/<tenant>/

[1.c] SELinux — graphroot labels (rootless Podman on RHEL)

Rootless Podman stores overlay layers under the user's graphroot — typically ~/.local/share/containers/storage. On standard home directories, SELinux labels this path correctly. However, when the home directory lives on a non-default mount point (/home/remote/, RAID, NFS, a custom partition), the graphroot inherits user_home_t instead of the expected data_home_t or container_var_lib_t label.

Containers that start from a user_home_t graphroot fail with:

• glibc (Debian/Ubuntu-based images): cannot apply additional memory protection after relocation: Permission denied

• musl/Alpine-based images: RELRO protection failed

These errors come from SELinux blocking mprotect() on the overlay filesystem — not from missing file permissions.

sfp server init --container-engine podman detects SELinux Enforcing mode automatically, resolves the graphroot path from podman info, and runs a best-effort restorecon to fix the labels. If restorecon succeeds, init continues normally.

restorecon may exit non-zero on some paths under the graphroot (notably overlay/*/diff directories) because rootless user-namespace UID remapping prevents the restorecon process from accessing files owned by mapped UIDs. This is expected — sfp server init runs restorecon -i to skip EPERM and missing-file errors on those overlay paths, then verifies the top-level graphroot label separately. Errors on the graphroot root directory itself are not ignored and will surface as an actionable error.

If the label persists as user_home_t after init, sfp throws an actionable error with the manual fix commands. Run these as root:

Derive the path from podman info:

podman info --format '{{.Store.GraphRoot}}'
# Example output: /home/remote/pdinh/.local/share/containers/storage
# The parent to relabel is: /home/remote/pdinh/.local/share/containers

Then relabel as root:

# Tell SELinux to treat the containers path like /var/lib/containers
sudo semanage fcontext -a -e /var/lib/containers \
  /home/<user>/.local/share/containers

# Apply the new context recursively
sudo restorecon -R -v /home/<user>/.local/share/containers

Verify the fix:

ls -Zd $(podman info --format '{{.Store.GraphRoot}}')
# Should show data_home_t or container_var_lib_t — NOT user_home_t

Images pulled before restorecon retain stale user_home_t labels on their overlay layers. After relabeling, prune all images and re-pull:

podman system prune -a -f --volumes
podman pull <your-images>

Skipping this step causes the same RELRO / mprotect failures even though the graphroot directory itself is now correctly labeled.

---

[2] Selecting Podman

Pass --container-engine podman to any sfp server command, or set the environment variable:

export SFP_CONTAINER_ENGINE=podman

When set, sfp-server:

• Uses podman for runtime commands (login, volume create, image prune)

• Uses podman compose (backed by Docker Compose v2) for stack lifecycle

• Fully qualifies Docker Hub image references (e.g. docker.io/library/postgres:...) in the generated compose file

• Adds :Z to bind-mount volumes for SELinux compatibility

---

[3] Compose provider

podman compose delegates to an external Compose provider. sfp-server requires Docker Compose v2 as that provider.

Provider	Supported?	Install
Docker Compose v2 (standalone binary)	✅ yes	See Setting up Docker on the server → Post-Installation Optional Steps
podman-compose (Python)	❌ no	Explicitly rejected by sfp-server

Verify after installing:

podman compose version
# Must show Docker Compose v2.x

---

[4] Troubleshooting

Symptom	Cause	Fix
podman compose: command not found or no compose provider found	Docker Compose v2 not installed as the Compose backend for Podman	Install the standalone docker-compose v2 binary — see [3] above. podman-compose is not supported.
Image pull fails with 401 Unauthorized or authentication required	Private sfp-pro registry credentials missing or expired	podman login source.flxbl.io — see [1.a] above; verify token has package: Read scope
Permission denied on bind-mount volumes	SELinux blocking container access to host paths	Verify --container-engine podman is set (enables automatic :Z relabeling). If still failing: chcon -Rt svirt_sandbox_file_t <path> — see [1.b] above
Docker works but Podman fails on the same compose file	Podman's Docker-compat layer has subtle differences (cgroup, network, rootless UID mapping)	Confirm --container-engine podman is set so sfp-server uses the Podman adapter (image qualification, :Z labels). Check podman info for rootless vs rootful mode; run podman compose up with --verbose for the specific error
cannot apply additional memory protection after relocation: Permission denied	SELinux user_home_t label on the Podman graphroot (rootless, non-default home mount)	Follow the manual fix in 1.c. Prune and re-pull images afterward.
RELRO protection failed (musl/Alpine images)	Same cause — SELinux blocks mprotect() on user_home_t overlay layers	Same fix as above. Alpine/musl reports the error differently but the cause is identical.
restorecon: unable to set context ... Permission denied on overlay diff paths during sfp server init	Rootless UID remapping prevents restorecon from accessing mapped-UID files	Expected — sfp server init skips EPERM on overlay diff paths and verifies the graphroot root label separately. If ls -Zd $(podman info --format '{{.Store.GraphRoot}}') shows data_home_t, the diff-path errors are harmless.
Containers still fail after running restorecon on the graphroot	Images pulled before relabeling retain stale user_home_t on their overlay layers	podman system prune -a -f --volumes then re-pull all images. See the warning in 1.c.