In the previous articles, we looked at how UI is losing its central role and how AI agents are becoming the new execution layer of products. Systems are no longer just responding to user input — they interpret intent, make decisions, and act. But once systems start acting, a different problem appears. Not what they can do. But how they fail.
Failure Doesn’t Look Like Failure Anymore
In traditional software, errors are usually visible. A flow breaks, a button doesn’t work, a request fails. The system makes it obvious that something went wrong.
Agent-driven systems behave differently. They don’t fail by stopping. They fail by continuing to produce outputs that look reasonable, sound convincing and still lead to the wrong outcome.
This pattern is well documented in system evaluations. Even advanced models can generate plausible but incorrect results, often with high confidence — especially in complex or multi-step scenarios.
The issue is not capability. The issue is that failure becomes harder to detect.
When the System Starts Deciding for You
As soon as an agent starts filtering information, prioritising options, or recommending actions, it is no longer neutral. It is making decisions on behalf of the user. That might mean excluding certain options as irrelevant, highlighting one path as “optimal,” or simplifying a complex situation into a single conclusion.
This pattern is already visible in real products. Early agent-based systems — from coding assistants to task automation tools — tend to perform well in clearly defined scenarios, but struggle when decisions involve trade-offs, context, or incomplete information.
The system doesn’t break. It produces an answer that looks correct, but is not grounded enough to be trusted. And when that happens, the user often doesn’t see what was removed from view. The mistake is not visible. It is embedded in the decision itself.
But this is not only a system-level problem. The quality of an agent’s decisions depends heavily on how the user directs it — the clarity of the prompt, the specificity of constraints, and the structure of the task. A vague instruction leads to a vague decision. An underspecified requirement gets silently dropped. Responsibility for invisible decisions is shared: the agent acts on what it is given, and the user shapes what it receives.
Norvana: Simplifying Without Losing Control
This tension becomes very clear in products that deal with complex and personal data. In Norvana, an AI-native health platform, the initial direction was familiar: organise everything into dashboards. Sleep metrics, stress indicators, activity data — all structured into categories that users could explore.
On paper, this looks like clarity. In practice, users either ignore most of the data or misinterpret it. More information does not lead to better decisions — it often leads to cognitive overload.
So the product moved away from dashboards and toward system-generated insights. Instead of exposing raw data, it began connecting patterns across signals and surfacing conclusions in context.
But this is exactly where the risk increases. Once the system starts interpreting health data, it is no longer just presenting information. It is making decisions. And those decisions can be wrong. The challenge was not removing the dashboard.
It was ensuring that, even without it, users could still understand, question, and adjust what the system was telling them.
Control Is Not Friction
There is a common instinct in AI products to remove anything that slows the system down. If the agent can act, why interrupt it? If it can decide, why expose complexity? But this way of thinking confuses control with friction.
Control is not about adding steps. It is about preserving the user’s ability to understand and influence what the system does.
This is why teams building agent systems increasingly focus on defining clear behavioural boundaries. When systems are allowed to act in external environments or make consequential decisions, those boundaries become critical — without them, autonomy quickly becomes unpredictable. And unpredictability breaks trust.
Visibility Is More Important Than Simplicity
One of the hardest lessons in Norvana was that simplifying the interface is not enough. You can remove dashboards, reduce visual noise, and present clean, high-level insights — and still end up with a system that users don’t trust.
Because what matters is not how simple the output looks. It’s whether the user understands where it came from. If the system highlights a pattern — for example, linking poor sleep with elevated stress — that relationship needs to feel grounded. The user doesn’t need every data point, but they need enough context to understand that the conclusion is based on something real.
This is a different kind of design problem. You are not just simplifying information. You are making reasoning visible.
Designing for Uncertainty
Another place where agents fail is how they handle uncertainty. Most systems default to presenting a single answer, even when the underlying data is incomplete or conflicting. That works in controlled scenarios, but it breaks in real-world usage.
Real decisions involve ambiguity. There are always trade-offs, unknowns, and competing signals. Designing for agents means acknowledging that uncertainty instead of hiding it. Systems need to communicate degrees of confidence, indicate where data may be weak, and, in some cases, present multiple plausible interpretations rather than a single definitive answer. This is not just a technical consideration. It directly affects how users trust the system.
Designing for Intervention
If users are no longer executing every step, their main role becomes intervention. They step in when something doesn’t feel right, when they want to adjust a decision, or when they need more control over the outcome.
This aligns with a broader shift toward human-in-the-loop systems, where AI performs actions but humans remain responsible for validating outcomes. That means intervention cannot be treated as an edge case. It has to be part of the core experience. Users need to be able to understand system assumptions, adjust them when necessary, and clearly see how those changes affect the outcome. Without this, the system either feels opaque or forces users back into manual work.
"Intervention is not a new burden. We have always operated this way. Think of it as hiring an assistant. You would not tell a new hire to send emails, make decisions, and act on your behalf from day one."
You review their work, give feedback, and gradually extend trust as they prove reliable. The same logic applies to agents. Intervention is not friction — it is delegation with oversight. The difference is that with a human assistant, you can watch them work. With an agent, you often only see the result. That is why the system must make the in-between visible: not to slow things down, but to keep the user in a position to judge.
Where This Breaks in Enterprise
These problems become more visible — and more costly — in enterprise environments. In controlled demos, agent systems often perform well. The inputs are clean, the tasks are well-defined, and the outcomes are easy to validate.
In production, the conditions are different. Data is incomplete or inconsistent. Decisions have financial, legal or operational consequences. Multiple stakeholders depend on the outcome. In that context, a system that is “mostly right” is not good enough.
If an agent makes a recommendation that cannot be explained, it will not be trusted. If it filters out information without visibility, it creates risk. If it cannot be adjusted or overridden, it will be bypassed entirely.
In enterprise, two requirements sit at the core of trust: predictability and auditability. Predictability means the client knows in advance which flow the agent will follow — not just that it will produce a good result, but that the path itself is known and sanctioned. Auditability means every decision the agent makes can be reviewed after the fact — why it chose one option over another, what data it relied on, and what it excluded. Without these two mechanisms, enterprise adoption stalls — not because the models are weak, but because the systems are not designed for the level of control that regulated, high-stakes environments demand.
This is why many enterprise AI initiatives stall. Not because the models are weak, but because the systems are not designed for control.
What This Means for Product Teams
Once agents become part of the product, the hardest problems are no longer in the interface. They sit in how the system behaves under imperfect conditions.
How it handles incomplete data. How it balances competing signals. How it communicates uncertainty. How it recovers when it is wrong.
These decisions are often invisible in the interface, but they determine whether the product works in the real world. This changes how products should be evaluated. Not by how much they automate, but by how well they remain understandable and controllable as they do.
What Comes After Behaviour
In the previous article, we argued that product design is moving from interface to behaviour. The next step is more specific. As systems take on more responsibility, behaviour alone is not enough. It has to be governable.
Across AI-native products, a new layer is emerging — not UI, not behaviour, but control. A layer that ensures that autonomous systems remain transparent, predictable, and adjustable. The interface becomes part of that layer, but it is no longer the centre of it. What matters is whether the system can be trusted to act — and whether that trust can be maintained over time. Because in real products, failure is inevitable.
The only question is whether it is visible, understandable and correctable when it happens. That is what separates a system that works in theory from one that works in practice.
What does this mean for you?
Discuss with your AI.
A Product Strategist with over 13 years of experience in marketing, product strategy, and branding. His love for analytics, funnels, and a structured approach ensures that the digital products we craft aren't just functional—they impress.
