One: Tool-call cascades
Tool-call cascades remain hard (level 4 of the Failure Pyramid) because the tool universe is not under our control. A third-party API changes its response schema; the agent does not detect the change; downstream tools compound the error. Our current mitigation is schema-validation on every tool response and pinning the schema version. The mitigation works when the tool gives us a schema version. Most do not.
Two: Context-window blow-out
Context-window blow-out (level 5) is partly mitigated by the 1M context window in Sonnet 4.7 but not fully solved. We have caught these in retrospective audits more often than we have caught them at runtime. Detection requires comparing summaries against ground truth, which is the kind of expensive verification step that is hard to justify on every run.
Three: Silent drift detection
Silent drift detection (level 1) is at ~80% catch-rate in our pipeline. The remaining 20% are outputs that look right, are internally consistent, and are quietly wrong. The architecture choices that get us higher are: cross-checking against last-known-good with a delta threshold, second-model evaluator on high-stakes content classes, reader-in-the-loop on flagged outputs. None of these are a complete solution. They are layers. We add layers as we find new failure shapes.
Why this is a useful Note to publish
The temptation in operator-credentialed publishing is to write only about what you have solved. The taxonomy of what we have not solved is more useful to a reader trying to evaluate their own pipeline. If your pipeline catches 90% of silent drift, you are ahead of us. If your pipeline catches 50%, you are behind. Knowing the number is the start of the work.
Read next
Oliver runs Digital Signet, a research and product studio that operates ~500 production sites with AI agents as the engineering layer. The Digital Signet portfolio is built using a continuous AI-agent build pipeline, one of the largest agent-operated publishing operations on the open web. The handbook draws directly from those deployments: real cost data, real failure modes, real recovery patterns.