stack-orchestration-and-global-setup

Full-Stack Orchestration and Global Setup

YouTeacher is not one server you can boot and point a test at. It is a fleet — a Next.js web app in front of separate auth, job, profile, talent, content, admin, discord, and games-BFF services, several of them paired with background workers, all sitting behind an nginx gateway. The integration harness exists to stand that whole thing up on one machine, prove every piece is alive, scrub any residue from the last run, and only then let a single Playwright process drive the browser through all of it. Its job is to make "the platform" behave like one address that either works end to end or fails loudly before a test does.

Two compose files, layered

The stack is described by two Docker Compose files stacked on top of each other. The base file brings up the application services and the gateway. A second, local-infrastructure file overlays it: it adds the stateful dependencies — PostgreSQL (one instance hosting several service databases), Redis, Meilisearch, and MinIO for object storage — and it rewrites each service's startup command to run a Prisma schema push before booting. A --remoteDB switch drops that second file, so the same application services can instead point at remote infrastructure. Layering keeps the definition of what the services are separate from the definition of where their data lives.

Host networking and the gateway

Every container runs on the host network rather than a private bridge. That means each service binds a fixed localhost port directly — auth, job, profile, talent, content, admin, discord, and the games BFF each own a port in the 3000-range — and the test runner reaches them at localhost with no Docker DNS in between. In front of them sits a single nginx reverse proxy, the gateway, which routes purely by URL path prefix: one prefix per service. The browser (and the tests) see one origin on the gateway's port; the request quietly fans out to whichever service owns that prefix. This mirrors production, where the same path-prefix routing sits behind the real edge proxy.

One concrete tuning lives here because host networking makes it matter. The web server's keep-alive timeout is set deliberately longer than the gateway holds idle upstream connections. Node's standalone server would otherwise close a pooled connection first; nginx, not knowing, would reuse that dead socket for the next request, hit a reset, and — because these are POSTs it will not retry — hand back a 502. Keeping the server's connections alive past the proxy's reuse window removes the race. It is the kind of detail that only surfaces when real services talk to a real proxy, which is the whole point of the harness.

Health as a staircase

Readiness is checked at three ascending levels, each stricter than the last.

  • Per-service healthchecks. Every application service declares a Docker healthcheck that fetches its own /health endpoint on an interval, with a generous start period for services that boot slowly.
  • Dependency gating. The gateway declares depends_on every service with a healthy condition, so nginx does not start routing until each backend reports healthy. Workers, which have no HTTP surface, gate on their parent service being healthy and are only required to have started.
  • Application-level verification. Once the gateway is up, Playwright's global setup runs its own pass: it waits for the gateway's aggregate /health to answer, then probes each service through the gateway at its real API path, retrying several times and treating any response other than a 502/503 as "up". This last layer proves not just that a container is running but that the gateway can actually reach it end to end — the exact path a test will use.

Global setup: start from a clean slate

Before any test runs, global setup scrubs state that a previous run could have left behind in the shared infrastructure — the sources of flakiness that have nothing to do with the code under test:

  • Meilisearch search indexes are emptied and the harness polls each delete task to completion, so no stale documents survive into a search assertion.
  • Background job queues (BullMQ, stored in Redis) are cleared, so no leftover job fires mid-test.
  • Rate-limit counters in Redis are wiped — an explicitly allowed exception — so a fresh run does not inherit throttling from the last one.

The ordering is deliberate: verify health first, then clear the data planes, then start driving. A test should meet a stack that is both alive and empty.

Global teardown: schema in the boot, not the setup

The lifecycle has a matching decision at the end. Tables are not dropped before the run — services create their own schema on startup via a Prisma push. Instead, global teardown, after all tests finish, clears Redis test keys and queues, empties the search indexes again, and drops every table in each service database. The schema is owned by the service's boot sequence; the harness only guarantees the databases are empty afterward. Combined with the version-stamped log files each service tees into a results directory, a completed run leaves behind its evidence and a clean floor for the next one.

Why it reads as one system

None of the pieces is exotic — layered compose files, host networking, an nginx path-router, per-container healthchecks, a setup/teardown pair. What makes the harness coherent is that they compose into a single guarantee: by the time a test's first line runs, the whole fleet is booted, individually healthy, reachable through the one gateway the test will actually use, and scrubbed of the previous run's data. The test author gets to think about one platform at one address, and everything underneath — the fan-out, the readiness staircase, the cleanup — has already been made true.

about this entry

One of sijie's wiki entries. The AI on this site is grounded in the same corpus and answers in sijie's voice, with citations back to entries like this one — answering costs sijie money, so it waits behind a code: enter an access code →