Product-owned upgrade: the instance rebuilds itself in place

Parent: key-designs

Verdict (2026-09-04): the owner presses upgrade inside the product (admin System panel, or MCP instance.upgrade); the backend writes one substrate-blind signal — a timestamp into a file on a shared volume — and stops. An updater sidecar, the only container holding docker.sock, sees the change, pulls the channel tag and recreates every sibling container in place from its own inspected config (the Watchtower model). The schema catches up because the backend applies its own migrations at boot. Nothing in the backend knows how it is deployed.

Motivating problem. In the image-based stack the backend deliberately has no docker.sock (deployment) — it cannot pull images or recreate its own containers. Until 2026-09-04 the button sent a request to an opaque URL the owner supplied (STANDMEET_REDEPLOY_HOOK), pushing deployment knowledge into the product and onto the owner's discipline. The first sidecar (3d1ce1aaf, same day) ran docker compose -p standmeet up against a fetched canonical compose — which hardcoded the project name (a deployment named differently got a parallel empty stack instead of an upgrade) and needed secrets as ${...} from an .env it could not reliably get (infra/updater/main.go:7-11 records both failures). Separately, before 2026-08-31 nothing ran backend/db/migrations/*.sql: an owner pulling a new image got code that wants a column the volume does not have — a backend that will not start. The schema lives in the volume, not the image; an upgrade that ignores that is not an upgrade.

The design

• One signal, no fork. instance.upgrade → SignalRedeployer.Trigger (backend/cmd/server/port/upgrade_signal.go:56-69): write a unix timestamp to <path>.tmp, rename over STANDMEET_UPGRADE_SIGNAL (atomic — the sidecar never reads a half-written file). The payload names no version: the compose's channel tag decides what is pulled, so version policy lives in one place. Configured() is only "the path is set"; the composition root builds exactly one redeployer (boot_upgrade.go:29-34), never a choice between adapters — a Coolify-shaped host would run a different sidecar reading the same file.
• instance.upgrade_check (internal/stats/ops/upgrade.go:107-123) asks the registry for the newest tag (port.NewReleaseChannel; default https://ghcr.io, cmd/server/config/config.go:156) and returns current / latest / comparable / available / can_apply. comparable=false for an unstamped dev build — "cannot compare" must never read as "up to date". Losing the registry costs latest, never current.
• The receipt is measured, not assumed. instance.upgrade returns only requested:true — the process answering is among those being replaced. The browser polls /api/v1/instance afterwards (app/src/lib/admin/use-upgrade.ts) and reports what happened, including "ran, version unchanged" (a pinned tag).
• Updater sidecar (infra/updater/, Go, 7a3a749e7): polls the signal every 5 s (main.go:37-40), acts only when the file content changes (main.go:64-79 — a sidecar restart never replays an old press). It inspects its own container to read com.docker.compose.project (docker.go:44-58) — it learns the real project name instead of being told — lists the siblings under that label, skips itself and any image outside STANDMEET_IMAGE_PREFIX, bumps each image's tag to STANDMEET_CHANNEL (bumpTag, main.go:110-120) and recreates: pull, stop, remove, create from the old container's inspected config / host config / networks with only the image changed, start (docker.go:91-129). Volumes, env (the secrets), names, labels, restart policy ride along untouched. The compose wires it: infra/deploy/docker-compose.yml:333-345 (signal path, channel, docker.sock).
• The schema follows the binary. pgstore.Migrate runs at backend/cmd/server/main.go:91, before serving. Migrations are embedded (go:embed, backend/db/embed.go) so code and schema are one artifact (internal/infra/pgstore/migrate.go:1-42); a schema_migrations ledger records what ran; every migration is reentrant (IF NOT EXISTS / guarded blocks), so there is no baseline branch — the first version had one, and on dev it marked a genuinely missing migration as applied. Fail = don't serve. Hence the rule: every schema change ships as a migration, and the upgrade path is tested against an old volume, not only the fresh-install path.

The release side

• Packages link to the repo via the OCI (Open Container Initiative) source label org.opencontainers.image.source (backend/Dockerfile:96, app/Dockerfile:14, infra/db/Dockerfile:30; ba219c391, 2026-09-01). Trap: a newly pushed ghcr package is private by default — an anonymous host cannot pull it, the whole deploy fails with no container and no log, the old version stays up; every new name in IMAGES must be flipped to public in the ghcr UI (there is no REST endpoint for it).
• Release gates (Makefile): release-push depends on secrets + secrets-image (:1444 — gitleaks over every built image; it refuses to run when gitleaks is missing rather than report success for work not done; Next.js framework build keys allowlisted, d846337bc). After the push: release-assert-version (:1369 — runs the pushed backend image's --version and compares to the tag; the stamp is -ldflags -X …port.appVersion, backend/Dockerfile:73), release-assert-stripped (:1400 — proves no data-testid survives in the release build's SDK widgets, Puck chunks exempt; bec5394fb, 2026-09-07), release-assert-multiarch (:1579 — docker manifest inspect must list amd64). These are mechanical-guardrails: each asks the artifact, not the author.
• Owner-side half — MCP client version-skew advisory (sdk/packages/core/src/version-skew.ts, 4442e8fe9, 2026-09-06). classifySkew(client, server, minCompatibleClient) → ok / warn (differs but above the floor: "run update_self") / incompatible (below the floor the server advertises — the transport/signing contract may have moved). Unparseable versions → ok, never nag on missing data. Pure, no I/O; wired in sdk/packages/mcp-client/src/bridge.ts. An instance upgrade therefore cannot silently strand the owner's client — the service-handle side of the same move.

The honest ceiling

• The updater never upgrades itself (main.go:91-93) — a new updater image lands on the next full stack restart.
• No sidecar → no button. The source-build stack (docker-compose.prod.yml) ships no updater; Configured() is false and the panel says the upgrade happens outside the instance and what to run. Stale string: the admin i18n key upgradeManual (app/src/i18n/messages/*/admin-shell.json:284, shown when can_apply is false — use-upgrade.ts:192) still tells the owner to "set STANDMEET_REDEPLOY_HOOK" — a knob the backend no longer reads (3c9fb6113 removed RedeployHookURL). The rest of the sentence is right; that clause is dead advice, unfixed as of 2026-09-07.
• One channel, forward only. bumpTag always moves to the channel tag; there is no in-product rollback — rollback is the owner re-pinning STANDMEET_IMAGE_TAG at the host. A compose with a pinned tag makes the button a no-op, which the receipt honestly reports.
• Sequential recreate, brief outage, no health gate. Siblings are recreated one by one in whatever order the container list returns; nothing waits for the database before the backend, nothing checks health between steps.
• Migrations are forward-only — no down scripts. A failed migration is a backend that will not serve, by design; recovery is manual.

Standing points

• The product emits a pulse, never a deploy command; substrate knowledge lives in the adapter. The owner's own host stays the only privileged party — chain-sovereignty.
• A version that is not the build's is worse than none (release-assert-version; the System panel of monitor shows it).
• Every schema change = a migration + an upgrade-* spec against an old volume: upgrade-pending-email-columns.spec.ts (old volume + deploy applies the schema; the same version again is a ledger no-op; no ledger + a missing migration → applied, not assumed; replaying never touches a hue the owner can set), upgrade-embed-schema.spec.ts, upgrade-application-code-unique.spec.ts, upgrade-code-entropy-compat.spec.ts (an already-issued short code still opens a session). The panel: admin-system-upgrade.spec.ts (the button really asks the registry; when the instance cannot apply, the button must not say "upgrade"). The sidecar's own run: infra/updater/updater-e2e.sh; fresh volume: infra/db/fresh-install-e2e.sh.

Built 2026-08-28 → 2026-09-07. f7c0b379d (2026-08-28, the button, and it says what it can do); bd45353f4 (2026-08-31, migrate at boot); ba219c391 (2026-09-01, OCI label); 3c9fb6113 + 3d1ce1aaf + 7a3a749e7 (2026-09-04: one signal; sidecar; in-place recreate); d846337bc (2026-09-04, secrets-image allowlist); 4442e8fe9 (2026-09-06, skew advisory); bec5394fb (2026-09-07, stripped gate). Design seeds: docs/design/product-owned-upgrade.md, docs/design/mcp-self-update.md — seeds, not evidence; the file:line citations above are.

Written 2026-09-07 against standmeet-new main 36789537d (v0.1.31).