Keepalive in two layers: is the machine there, is the program working
A long-lived idle connection cannot tell you whether the other end still exists. "Alive" is actually two different questions, and each layer of keepalive answers one of them — neither can do the other's job.
Two ways to be dead. (1) The machine or the path is gone: host crashed, network cut — plus the silent variant, the NAT mapping for the connection expired (see how-nat-works), so the path quietly stopped existing. (2) The machine is fine but the program is not: process wedged, deadlocked, event loop blocked. The OS is healthy; the application does no work.
Layer one — TCP keepalive — checks machine and path. The kernel, after an idle period, sends a probe; the peer's kernel answers. That proves the host and the route exist. It cannot see failure mode (2): a completely stuck process still gets its probes answered by its kernel, so the connection looks alive while the application is dead. Its other wart: the default first probe fires after two hours of idleness — applications that rely on it tune this down explicitly.
Layer two — application heartbeat — checks the program. The application itself exchanges small frames on a timer (WebSocket ping/pong, HTTP/2 PING, gRPC keepalive). Only the peer application can answer, so a reply proves its event loop is turning and it can take work. This layer does a second job: the periodic traffic keeps NAT mappings and load-balancer idle timers from expiring the connection. WireGuard's 25-second persistent keepalive exists mostly for this second job.
Where it works. Database connection pools (an idle pooled connection killed overnight by a middlebox is handed out as if healthy; pool validation and tuned keepalive exist to evict it first). Cloud load balancers (a fixed idle timeout, enforced silently — a heartbeat interval longer than it kills connections without telling anyone). Mobile messaging (heartbeats detect tunnel/Wi-Fi-to-cellular drops and drive reconnect; presence indicators — the green "online" dot — are literally this layer made visible). gRPC between services (both sides' ping settings must agree; a too-eager client gets its connection cut for pinging too much). IoT fleets (the heartbeat doubles as the device registry's freshness signal).
Balancing the interval — three constraints, then three knobs. Detection delay ≈ interval + death window, and cost ≈ size × frequency × fleet. The interval is not chosen freely: (a) the least patient middlebox on the path caps it — the heartbeat must beat the shortest idle timeout (a NAT mapping, the LB's limit), or the connection dies silently regardless; (b) the business's tolerance for late detection sets the target — presence can be a minute late, leader failover cannot be ten seconds late; the stricter of (a) and (b) wins; (c) cost at fleet scale — a million devices at 30 s is ~33k heartbeats per second of pure overhead, which is why phones merged all apps into one system push channel. Inside those constraints: require 2–3 consecutive misses before declaring death (one miss is usually weather, not death); add ±10–20% random jitter so a fleet does not beat in lockstep and flood the server every interval; and keep reconnect backoff separate from the heartbeat — a recovering server should not be trampled by its whole fleet reconnecting at once. Rule of thumb: interval at 50–80% of the path's shortest idle timeout, death after 2–3 misses, jitter on, backoff on reconnect.