Retry has one owner: the job layer

Parent: events

Status: released in v0.1.76 (2026-09-27) — design and as-built record in docs/design/event-bus-outbox-webhooks.md in the StandMeet repo.

Only the job layer retries. A handler only classifies a failure; the policy declared on the job kind decides when to try again.

Before vs after

3 × 3 × 3 = one failure became up to 27 requests. Now only the job layer retries:

internal/infra/retry and notifyPolicy are deleted. The gate check-retry-only-in-jobs.sh forbids retry.Do outside internal/infra/jobs (see no-bypass-by-structure).
Webhook delivery runs httpx with NoRetry.
The Meili client's own retries are disabled (meilisearch.DisableRetries()); corpus.index jobs retry instead.

Synchronous calls are sent once

A call a visitor waits on (calendar free/busy, insert and delete through the openapi adapter) no longer runs retry.Do inside the request. A transient failure maps to ErrCalendarUnavailable at once. Reason: one retry owner, and a visitor-facing call answers fast. A retry that matters, such as the compensating calendar delete, is a durable supplier.invoke job.

Failure classification: what a handler returns

Every 4xx except 408 and 429 is permanent. Retrying would not help and would only hammer the receiver.
A snooze does not consume an attempt. A webhook snooze from Retry-After is capped at 10 h.
On River, jobs.Discard(err) is a cancel-with-error and shows as discarded; a cancel from the panel shows as cancelled.

Policy (declared on the job kind, as data)

Kind	Max attempts	Backoff	When exhausted
`corpus.index`, `corpus.reindex`	10	`DefaultBackoff`: from 1 s, doubling, capped at 15 min	`discarded` + alert; the panel can retry
`owner.notify`, `access_request.approval_mail`, `owner.email_confirmation`	8	20 s, then ×3 per attempt: about 6 h	`discarded` + alert (on the panel, not another email)
`webhook.fanout`	10	`DefaultBackoff`	`discarded` + alert
`webhook.deliver`	18	Svix: 5 s · 5 min · 30 min · 2 h · 5 h · 10 h · 10 h… (about 5.3 days)	`discarded`; the endpoint is disabled after 5 days of continuous failure
periodic jobs	no retry	the next period	the panel shows the last failure

Every backoff adds ±10% jitter, so a batch of jobs that failed together does not retry at the same instant.

Do not hammer a dead endpoint

Any non-success sets failing_since (410 and 429 too); a success clears it.
At most one delivery per endpoint is in flight, through a lease row (busy_until); see concurrency-control.

Retries with side effects: can it send twice?

Webhook: every retry carries the same webhook-id (the event id). Consumers dedupe on it. See webhooks.
Mail: SMTP has no idempotency key. The order is "send, then mark" (notified_at, replied, or the deleted booking notice); a crash in between sends a duplicate. That is the price of at-least-once, far better than "lost on failure". The message carries Message-ID <event id@standmeet>, and most mailboxes merge duplicates by it.
Mail throttle: the per-recipient cap (30 per hour) snoozes to the next window instead of dropping. The owner-notify burst cap (5 per owner per hour) stays a deliberate drop; see message-loss-guarantees.
Search index: an upsert, naturally idempotent.

Manual retry

The tasks-panel retries a single job; webhooks.redeliver re-delivers every discarded job of one endpoint after the receiver is fixed. Covered: the Phase 2 "sink returns 500 twice" case, 429 with Retry-After → retried later without consuming an attempt, and 410 → discarded immediately.