Evaluating agents in production: quality and failures

Imagine an artificial intelligence agent responsible for handling customer requests. It queries the systems correctly, responds in a few seconds, and shows no technical errors during execution. However, it skips a step needed to solve the problem.

From the standpoint of traditional monitoring, everything may look normal. But for the customer and for the company, the task was not completed.

This example illustrates a challenge that grows in importance as AI agents take on more complex activities in organizations. An agent can work technically without necessarily delivering the expected result.

In June 2025, Gartner projected that more than 40% of agentic AI projects could be canceled by the end of 2027, due to rising costs, difficulty demonstrating business value, or insufficient risk controls [1].

This is a projection, not a cancellation rate already observed. Even so, the indicator reinforces the importance of continuously evaluating the reliability and results of agents, especially as they move from experimentation to production.

What it means to evaluate an AI agent

Evaluating AI agents goes beyond checking whether an answer is correct or whether a given process finished without errors.

Unlike conventional applications, agents can interpret goals, select tools, query information, and run sequences of actions that vary with context. That is why two runs of the same task may follow different paths.

The Microsoft Foundry documentation distinguishes evaluating outcomes from analyzing the steps the agent performed. Among the criteria considered are task completion, adherence to instructions, and appropriate use of the available tools [2].

That distinction matters because a seemingly correct answer does not necessarily mean the agent followed the expected process.

Consider an agent in charge of updating registration data. It may confirm to the user that the request was fulfilled even though it never made the change in the system. In that case, the quality of the answer and the actual completion of the task produce different results.

Evaluating an agent requires checking not only what it answered, but also what it actually did.

That involves observing the quality of the delivery, compliance with instructions, and the appropriateness of the actions performed. The greater the autonomy granted, the more relevant these criteria become, as we saw in human-in-the-loop and autonomy.

How to identify failures that do not show up in the final answer

Some problems are relatively easy to identify, such as a system call that returns an error or an interrupted execution.

Others are less obvious. An agent may use an inappropriate tool, query outdated information, repeat unnecessary steps, or misinterpret the result of an operation.

These situations do not always produce errors visible to the user, but they can compromise the quality, security, or efficiency of the process.

In June 2026, IBM published an analysis on testing AI agents, highlighting indicators such as task success rate, accuracy in tool use, and evaluation of the execution trajectory [3].

Trajectory evaluation makes it possible to examine the sequence of actions performed by the agent, identifying whether it used the right resources and followed a path consistent with the established goal.

This type of analysis complements agent observability and is especially important when agents perform activities across different corporate systems.

An agent responsible for processing a financial request, for example, may correctly query the necessary data but use inappropriate parameters when executing an operation. The system may record a successful execution even though the result produced does not match what was expected.

The absence of a technical error is not synonymous with quality, compliance, or correct task completion.

That does not mean every execution must follow exactly the same sequence. Depending on the activity, different paths can produce equally valid results.

What matters is to establish criteria that make it possible to tell acceptable variations from inappropriate behavior, considering the goal of the task, the applicable policies, and the expected results.

Why evaluation needs to continue in production

Approving an agent during development does not guarantee that its behavior will remain adequate after deployment.

Changes in AI models, instructions, available tools, or the information used can alter results, even when the original goal stays the same.

Google Cloud distinguishes evaluating the final response from analyzing the execution trajectory, offering mechanisms to compare agent behavior against previously defined criteria [4].

This approach helps identify regressions, situations in which changes in the environment or configuration cause a decline in previously observed performance.

However, evaluating agents in production also requires considering the business context.

An agent may show a high task completion rate but produce results that often need to be corrected by human professionals. Another may carry out activities correctly but consume excessive resources or increase total handling time.

In those cases, technical indicators need to be analyzed together with quality and operational efficiency.

The completion rate, for example, can show how many requests were finalized. But it is also necessary to understand how many were resolved correctly, how many required rework, and whether the result met the needs of the process.

An agent’s performance should not be measured only by the number of tasks executed, but by the quality and usefulness of the results delivered.

This perspective helps companies identify improvement opportunities, set appropriate limits of autonomy, and decide which agents are ready to take on greater responsibilities.

Gartner reinforced this concern in October 2026, highlighting that expanding autonomy should follow the reliability agents have demonstrated, with continuous evaluations of consistency, traceability, predictability, and security [5].

The question, therefore, is not only putting AI agents into production, but developing the ability to track their quality and correct inappropriate behavior over time.

From vision to practice: ARGUS

As AI agents begin to take part in corporate processes, tracking their behavior and actions becomes essential to assess the reliability of the operation.

ARGUS, Target Solutions’ Operational and Agentic Intelligence platform, uses its AgentOps capabilities to support the monitoring, observability, and governance of AI agents.

The platform offers visibility into agent execution, their interactions, and resource consumption, helping identify unexpected behavior, investigate failures, and track operational indicators.

By connecting agents, people, and tools in a shared operational context, ARGUS also helps teams better understand how activities are carried out across different systems and processes.

That visibility provides important information to support the continuous evaluation of agents, without replacing the quality criteria and business results that each organization needs to establish.

Learn about ARGUS and see how it works in practice.

About the author

Paulo Florêncio de Menezes is an entrepreneur, partner, and co-founder of companies such as Target Solutions, BiPTT, and ARGUS.

He holds a degree in Data Processing from the Federal University of Amazonas (UFAM), an MBA in Business Management from Fundação Dom Cabral (FDC), and a Professional Master’s in Economics from Fundação Getulio Vargas (FGV).

In his Master’s research, he developed the IBEOP-IA (Brazilian Index of Potential Occupational Exposure to Artificial Intelligence), investigating the potential exposure of Brazilian occupations to AI capabilities by combining occupational data, artificial intelligence patents, and natural language processing techniques.

References

[1] Gartner. Gartner Predicts Over 40% of Agentic AI Projects Will Be Canceled by End of 2027. June 25, 2025.

[2] Microsoft. Agent Evaluators for Generative AI. Microsoft Foundry. Documentation on evaluating outcomes, adherence to instructions, and tool use by AI agents.

[3] IBM. Teste de agentes de IA: estratégias, métricas e melhores práticas. June 25, 2026.

[4] Google Cloud. Evaluate Gen AI Agents. Documentation on evaluating final responses and execution trajectories of agents.

[5] Gartner. AI Agent Success Requires Reliability Before Autonomy. October 2, 2026.