BuildingAI Agentsthat don't break.
An interactive field guide to the agents that actually survive contact with production: why they fail, how to build them, and how to prove they work. Scroll to explore.
The dream
being sold.
AI agents are being pitched as systems that don't just answer questions, but take actions: call tools, update systems, follow workflows, and push work forward. That's the dream.
Teams are betting on agents because they can save a lot of time. Agents can triage a support ticket, investigate an issue, draft a change, write and fix code, query a database, or trigger the next step in an ops workflow.
But most agent projects still don't survive contact with production. An MIT / Project NANDA report found that around 95% of gen-AI initiatives aren't producing measurable business impact.
Not because the idea is bad, but because reliability is hard. Tool calls fail. Context is messy. Costs and latency spike. Behavior changes across model updates. Debugging "why it did that" is often painful.
The current
state of agents.
An agent is a software system capable of making decisions, reasoning, and taking actions to perform complex tasks with reduced human intervention. At a very basic level, it looks like this:
When you're actually building an agent, think in terms of agent design, or flow. Many agent systems are variations of the same idea: the agent receives context, reasons about what to do next, and produces a response or action.
Depending on the design, this may involve keeping track of state or memory, planning over multiple steps, or interacting with external systems via tool calls, but none of these are strictly required. Some agents are purely reactive; others layer on structure as complexity grows.
Practical agents add layers: orchestration, tool reliability, verification, guardrails, observability. Hover any node to see its role.
Do you really
need an agent?
An agent is simply a system that can observe context, decide, and act on its environment. That doesn't automatically make it the right abstraction. The real question is whether adding autonomy actually helps or just adds complexity.
Agent-style autonomy is useful when a task requires ongoing decision-making across multiple steps, involves interaction with external systems, and has clear signals of success or failure.
If a fixed workflow plus one or two model calls is enough, the goal is hard to evaluate, or actions are high-risk without strong guardrails, adding an agent usually makes the system harder, not easier.
When in doubt, start with a tool-using assistant, instrument it, and only graduate to an agent once you consistently see real multi-step decision-making emerge in practice.
How companies use them
for business value.
Across companies, agents create business value in a few recurring categories. At ACME, their agent helps operators explore performance data, reason across datasets, choose tools, and follow investigative workflows, but never operates unchecked.
Investigative & analytical workflows
Agents answer complex questions that require multiple steps: querying data, aggregating results, checking assumptions, explaining conclusions. Value comes from shrinking "question" to "actionable insight," not removing humans entirely.
Triage & decision support
In support, security, and incident response, agents classify issues, gather context, suggest next steps, and escalate when needed. They reduce cognitive load; they don't close the loop autonomously.
Controlled automation behind feature flags
Some teams let agents take actions: updating records, triggering workflows, drafting changes, but only behind strong guardrails. Feature flags, permissions, and approval steps gate autonomy.
Internal productivity for eng & ops
Agents investigate issues, draft code changes, run diagnostics, or coordinate multi-step ops tasks. Tighter feedback loops make these environments ideal for earning trust before exposing agents to customers.
What these cases share is restraint. The agents that deliver value today are not "general workers," they're specialists. Scope narrow, goals clear, behavior observable and testable. When teams answer honestly what decision-making loop is expensive, repeatable, and well-defined enough for an agent to help, agents start producing real business impact.
Agents are brittle. Any small change can cascade into behavior you didn't expect. We needed a way to test the workflow itself, not just whether an answer looked right. Without simulations, we wouldn't be able to check all the permutations of features and user behavior.
AI Lead ArchitectACME
Problems
with agents.
After reading this, you're probably excited about AI agents. That excitement makes sense, but most break down quickly in practice.
Long-term planning is hard
LLMs are good at short-range reasoning but struggle to carry a coherent plan across many steps. Agents lose track of goals, repeat actions, or loop.
Massive solution space
At every step there are many plausible actions. Because they're non-deterministic, agents may choose a different path each time, so outcomes become hard to reproduce.
State & memory are unreliable
Context windows are limited; summaries lose detail. The agent's internal view drifts from reality, leading to repeated work or confidently wrong actions.
Fragile to tooling failures
The same task can succeed once and fail the next. When tool calls are malformed or return unexpected outputs, agents rarely recover gracefully.
Agents are not broken, but they are not magic either. These limitations are inherent, and ignoring them is why most agent systems never see real-world use. This is also where specification, cleanly defining what an agent should do and must not do, becomes the real bottleneck.
Major agent frameworks
you should know.
You technically don't need a framework. A simple loop around a model call, some state, and a few tool calls goes surprisingly far. Frameworks exist because once agents grow, the same problems keep appearing.
LangGraph
Agents as graphs and state machines. Explicit rather than implicit.
Built around the idea that agent behavior should be explicit: nodes represent steps in reasoning or action, edges define transitions. Strong fit for complex, multi-step workflows where predictability and debuggability matter.
Which is right for you? There is no single best answer. A rule of thumb: choose the simplest tool that lets you reason clearly about your agent's behavior today, while leaving room to add structure tomorrow. Frameworks should support your understanding of the system, not replace it.
The Agent Testing
Pyramid.
A pattern developed by Rogerio Chaves (CTO, LangWatch). Inspired by the traditional software testing pyramid, adapted to the realities of AI agents. All three layers are necessary; skipping one creates blind spots.
Simulations
End-to-end behavioral tests. Simulated users drive multi-turn flows; a judge evaluates whether the agent completes the task as a user experiences it.
Agent Observability
and Evals.
Once your agent runs in the real world, the question is no longer "can it respond?" It's "can I trust it, every single time, under messy real-world conditions?"
Output quality
Is the agent's response correct, complete, grounded, and useful?
Behavior
Did it call the right tools at the right time? Ask clarifying questions? Respect constraints?
Systems tradeoffs
Acceptable quality within latency, tool reliability, and cost budgets?
Scenarios:
simulated reality.
Unit tests and per-call evals aren't enough. You need to test how the agent behaves across whole conversations. Scenarios simulate users driving multi-turn flows, while a judge verifies real-world outcomes.