# Troubleshooting Common issues when setting up and running sfp server. *** ## General | Symptom | Cause | Fix | | ---------------------------------- | ---------------------------------------- | ------------------------------------------------------------------------------------------------ | | `denied: denied` from Docker login | Stale token, wrong registry | `docker logout && echo "" \| docker login -u --password-stdin` | | Domain returns NXDOMAIN | DNS not propagated or `A` record missing | `nslookup `; verify `A` record points at server; allow up to 48h propagation | | Need raw container logs | Compose project name matches tenant | `cd /opt/sfp-server/tenants/ && docker compose -p logs -f` | ## TLS / Certificates | Symptom | Cause | Fix | | --------------------------------- | ---------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ | | Caddy logs `cert file not found` | TLS cert not placed or not decoded | Re-run init with `ORIGIN_CERT`/`ORIGIN_KEY` secrets, or manually place `origin.pem` + `origin-key.pem` in `{tenantDir}/certs/` | | Let's Encrypt fails to issue cert | DNS not publicly resolvable or port 80 blocked | Verify `dig ` returns server IP from the public internet; open port 80 inbound for ACME challenge | | Certificate format error | Cert not PEM-encoded or not base64 | Certs must be PEM format; `.crt`/`.key` files are typically already PEM — rename and base64-encode: `base64 -w 0 origin.pem` | ## Services | Symptom | Cause | Fix | | ------------------------------------- | -------------------------------------- | ------------------------------------------------------------------------------------------------------------------- | | `external volume "…" not found` | Volumes not created yet | Use `sfp server start` (not raw `docker compose up`); it creates external volumes automatically | | Server healthy but `curl` returns 502 | App server not ready yet | Wait 30–60s after start; check `sfp server logs --service app`; Caddy shows a maintenance page until backend is up | | Supabase containers not starting | Missing compose profile | `sfp server start` activates the `supabase` profile automatically; raw `docker compose up` does not | | `VaultBootstrapService` fetch failed | Supabase auth not ready at server boot | Restart the server container: `docker compose -p restart server`; auth containers need \~15s to initialize | ## SSH | Symptom | Cause | Fix | | -------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `sfp server init`: `SSH connection error: Timed out while waiting for handshake` then `Docker is not installed or not available` | Target port 22 unreachable (firewall, security group, bastion-only network). The "Docker" line is misleading — read the SSH line above it. | Open a local tunnel via the bastion: `ssh -fNT -L 2222::22 @`, then `sfp server init --ssh-connection @127.0.0.1:2222 --identity-file ~/.ssh/`; tear down with `pkill -f "ssh -fNT -L 2222"`. | | `sfp server init`: `getaddrinfo EAI_AGAIN ` | `--ssh-connection` was passed an `~/.ssh/config` Host alias. `sfp` uses Node `ssh2`, which does not read `~/.ssh/config` — `Host`, `ProxyJump`, `IdentityFile`, `User` are all ignored. | Pass a literal `user@host[:port]` plus `--identity-file `. For `ProxyJump`-style routing, use the local tunnel recipe in the row above. | | `sfp server init`: `Failed to read private key: ENOENT ` | `--identity-file` path missing or unreadable by the user running `sfp`. | Pass an absolute path: `--identity-file /home//.ssh/`. `~/` expands; `$HOME` does not. | | `sfp server init`: `No authentication method provided` | Neither `--identity-file` nor `--passphrase` passed. `sfp` does not fall back to `ssh-agent` or `~/.ssh/id_rsa`. | Always pass `--identity-file ` with `--ssh-connection`. No `--use-agent` flag exists. | ## Authentication | Symptom | Cause | Fix | | ---------------------------------- | --------------------------------- | ------------------------------------------------------------------------------------------ | | GitHub OAuth callback URL mismatch | Callback URL doesn't match domain | Must be exactly `https:///auth/v1/callback` — check GitHub OAuth App settings | ## Cloud Supabase | Symptom | Cause | Fix | | --------------------------------------------------- | -------------------------------------- | ----------------------------------------------------------------------------------------------------------------- | | `init` cannot reach Supabase Cloud | Project IP allowlist or rotated key | Verify allowlist covers server IP; `curl -X GET "/rest/v1/" -H "apikey: "` to probe; check service key | | `network is unreachable` for IPv6 (`[2a05:d014:…]`) | Server has no IPv6 (common on Hetzner) | Switch `SUPABASE_DB_URL` to **Session pooler** (port 6543) from Supabase Dashboard → Connect | | Same as above (alternatives) | Same | Buy Supabase IPv4 add-on; or enable host IPv6 | *** ## Security best practices | Area | Action | | ---------------- | ----------------------------------------------------------------------------------------------------- | | Firewall (UFW) | `sudo ufw allow 22,80,443/tcp && sudo ufw enable` — SSH + HTTP redirect + HTTPS only | | Host updates | Enable unattended security updates | | Secret rotation | Rotate `DOCKER_REGISTRY_TOKEN` + Supabase service keys quarterly | | Log shipping | Ship server logs to a central store (don't grep on the box) | | Tenant backups | Snapshot `/tenants//` — covers `.env`, generated compose, admin `credentials.json` | | Database backups | Snapshot Supabase (cloud or self-hosted Postgres) on its own schedule — independent of tenant files | | Admin dashboards | Restrict ports 8080, 3100, 4873 to admin IPs via `ALLOWED_IPS` in `.env`; restart Caddy after changes | --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://docs.flxbl.io/flxbl/sfp-server/setting-up/setting-up-sfp-server/troubleshooting.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.