Runtime guardrails — the deterministic checks moved into the production path
Testing's twin: instead of (only) checking outputs in an eval suite, inline the deterministic checks into every production execution and act on failure immediately — block, retry with feedback, or degrade to a safe fallback. Frameworks: Guardrails-AI (validator library around model calls: schema, PII, profanity, groundedness), NVIDIA NeMo Guardrails (dialog-level rails: topic bounds, tool-permission rails, jailbreak deflection).
The standard rail stack per call: input rails (injection screening, topic/permission bounds before the model sees the text) → output rails (schema validity, banned content, citation-resolves, PII) → tool rails (which tools this flow may invoke, argument bounds — the capability wall) → on failure: auto-retry with the validator's error fed back, or refuse. Latency budget forces rails to be cheap and deterministic — the LLM-judge tier mostly stays offline/async (online-evals).
The conceptual point: rails acknowledge that eval-time confidence doesn't transfer to run-time for stochastic programs — every execution is a fresh draw, so every execution gets checked. Contrast with Software 1.0, where passing tests once is (mostly) proof forever.
um relation: um had this instinct before meeting the frameworks — law 1 IS a runtime rail ("the publisher does not trust a passed claim — it RE-RUNS the lint on the committed entry at publish time"), and the browser-drive capability split (#7: only the publisher holds the logged-in profile) is a tool rail made physical. What um can adopt: input rails — law 4 declares platform text data-not-instruction, but nothing mechanically screens/delimits inbound text at the context-assembly boundary yet beyond the documented delimiter convention; a deterministic injection-pattern screen at L0/L1 ingestion would be the rail form of law 4 (and automated-red-teaming is how you'd test that rail).
Source: evals mainstream practice (Guardrails-AI / NVIDIA NeMo Guardrails, 2026-07 landscape check)