Getting AI agents to perform reliably in production – not just in demos – is becoming more difficult than enterprises expected. Fragmented data, unclear workflows, and uncontrolled growth rates are slowing down deployment across industries.
“Technology often works well in demonstrations,” said Sanchit Veer Gogia, chief analyst at Greyhound Research. “The challenge begins when one is asked to work within the complexity of a real organization.”
Burley Kawasaki, who oversees agent deployment at Creatio, and the team developed a methodology based on three topics: data virtualization to work around data lake latency; Agent dashboards and KPIs as a management layer; and tightly bound use-case loops leading to higher autonomy.
In simple use cases, Kawasaki says these practices have enabled agents to handle 80-90% of tasks themselves. With further tuning, they estimate they can support autonomous solutions in at least half of the use cases, even in more complex deployments.
“People are doing a lot of experiments with proof of concepts, they’re doing a lot of testing,” Kawasaki told VentureBeat. “But now in 2026, we are starting to focus on mission-critical workflows that either drive operational efficiencies or additional revenue.”
Why do agents keep failing to produce?
Enterprises are eager to adopt agentic AI in some form – often because they fear missing out before they can even identify tangible real-world use cases – but face significant hurdles around data architecture, integration, monitoring, security and workflow design.
The first hurdle is almost always related to data, Goggia said. Enterprise information rarely exists in a clean or integrated form; It spans SaaS platforms, apps, internal databases, and other data stores. Some are structured, some are not.
But even when enterprises overcome the data recovery problem, integration is still a major challenge. Goggia pointed out that agents rely on APIs and automation hooks to interact with applications, but many enterprise systems were designed long before such autonomous interactions became a reality.
This may result in incomplete or inconsistent APIs, and the system may respond unexpectedly when accessed programmatically. Organizations also run into problems when they try to automate processes that were never formally defined, Goggia said.
“Many business workflows depend on secret knowledge,” he said. That is, employees know how to resolve previously observed exceptions without explicit instructions – but, when the workflow is translated into automation logic, those missing rules and instructions become surprisingly obvious.
tuning loop
Kawasaki explained that Createio deploys agents in a “limited scope with clear guardrails”, followed by an “explicit” tuning and validation phase. Teams review initial results, adjust as needed, then retest until they reach an acceptable level of accuracy.
That loop usually follows this pattern:
- Design-Time Tuning (before go-live): Performance is improved through early engineering, context wrapping, role definitions, workflow design, and grounding in data and documents.
-
Human-in-the-loop improvements (during execution): Devs approve, edit, or resolve exceptions. In instances where humans have to intervene the most (escalation or approval), users establish stronger rules, provide more context, and update workflow steps; Or, they’ll limit tool access.
-
Ongoing Optimizations (After Go-Live): Devs continue to monitor exception rates and outcomes, then repeatedly tune as needed, helping to improve accuracy and autonomy over time.
Kawasaki’s team applies retrieval-enhanced generation to ground agents in enterprise knowledge bases, CRM data and other proprietary sources.
Once agents are deployed in the wild, they are monitored with a dashboard providing performance analytics, conversion insights, and auditability. Essentially, agents are treated like digital workers. They have their own management layer with dashboards and KPIs.
For example, an onboarding agent will be included as a standard dashboard interface that will provide agent monitoring and telemetry. It is part of the platform layer – orchestration, governance, security, workflow execution, monitoring and UI embedding – that sits "Above LLM," Kawasaki said.
Users see a dashboard of the agents in use and each of their processes, workflows, and results executed. They can “drill down” into an individual record (such as a referral or renewal) that shows step-by-step execution logs and related communications to support traceability, debugging, and agent tweaking. Kawasaki said the most common adjustments include logic and incentives, business rules, quick reference and tool access.
Biggest issues faced after deployment:
- The amount of exception handling can be high: Initial spikes in edge cases often occur until guardrails and workflows are tuned.
-
Data Quality and Completeness: Missing or inconsistent fields and documents can cause stress; Teams can identify which data to prioritize for grounding and which checks to automate.
-
Audit and Trust: Regulated customers, in particular, require clear logs, approvals, role-based access controls (RBAC), and audit trails.
“We always explain that you have to allocate time to train agents,” Ekaterina Kostereva, CEO of Createo, told VentureBeat. “When you switch to the agent it does not happen immediately, it needs time to understand it completely, then the number of mistakes will be reduced.”
"data readiness" Overhaul is not always required
When deploying agents, ask “Is my data ready?” is a common initial question. Enterprises know that data access is important, but it can be put off by a large-scale data consolidation project.
But virtual connections can allow agents to access underlying systems and avoid typical data lake/lakehouse/warehouse delays. Kawasaki’s team built a platform that integrates with the data, and is now working on an approach that will pull the data into a virtual object, process it, and use it like a standard object for UI and workflow. This way, they do not need to “maintain or duplicate” large amounts of data in their database.
This technology could be helpful in areas like banking, Kawasaki said, where transaction volumes are too large to copy into a CRM, but “are still valuable for AI analysis and triggers.”
Once integration and virtual objects are established, teams can evaluate data completeness, consistency, and availability, and identify low-friction starting points (such as document-heavy or unstructured workflows).
Kawasaki stressed the importance of “really using the data in the underlying systems, which is really the cleanest or the source of truth anyway.”
matching agents for work
Best suited for autonomous (or near-autonomous) agents are high-volume workflows with “clear structure and controllable risk,” Kawasaki said. For example, document intake and verification in onboarding or loan preparation, or standardized outreach such as renewals and referrals.
“Especially when you can tie them to very specific processes inside an industry – that’s where you can really measure and deliver hard ROI,” he said.
For example, financial institutions are often siled by nature. Commercial lending teams work in their own environment, wealth management in another. But an autonomous agent could, for example, look across different departments and different data stores to identify commercial clients who might be good candidates for wealth management or advisory services.
“You would think it would be an obvious opportunity, but no one is looking at all the silos,” Kawasaki said. Without naming specific institutions, he claimed, some banks that have implemented agents in this scenario have seen “millions of dollars of incremental revenue gains.”
However, in other cases – especially in regulated industries – agents with longer references are not only preferable, but necessary. For example, in multi-step tasks such as gathering evidence across systems, summarizing, comparing, drafting communications and preparing arguable arguments.
“The agent is not giving you any immediate response,” Kawasaki said. “It can take hours, days, to complete the entire end-to-end tasks.”
This, he said, requires streamlined agentic execution rather than a “single giant signal.” This approach divides the task into deterministic steps to be performed by sub-agents. Memory and context management can be maintained in different phases and time intervals. Grounding with a RAG can help keep output tied to approved sources, and users have the ability to direct expansion to file shares and other document repositories.
This model typically does not require custom retraining or a new foundation model. Whatever model enterprises use (GPT, cloud, Gemini), performance is improved through signaling, role definition, controlled devices, workflows and data grounding, Kawasaki said.
He said the feedback loop puts “extra emphasis” on intermediate checkpoints. Humans review intermediate artifacts (such as summaries, extracted facts, or draft recommendations) and correct errors. They can then be transformed into better rules and retrieval sources, narrower tool scopes, and better templates.
“What’s important for this style of autonomous agent is you combine the best of both worlds: the dynamic logic of AI, with the control and power of true orchestration,” Kawasaki said.
Ultimately, agents need coordinated changes to enterprise architecture, new orchestration frameworks and clearer access controls, Goggia said. Agents must be assigned identities to limit their privileges and keep them within limits. Observability is important; Monitoring tools can record task completion rates, escalation events, system interactions, and error patterns. This type of evaluation should be an ongoing practice, and agents should be tested to see how they react when faced with new scenarios and unusual inputs.
“The moment an AI system can take action, enterprises will have to answer many questions that rarely come up during co-pilot deployments,” Goggia said. Such as: What systems is the agent allowed to access? What kind of work can it do without approval? Which activities should always require human judgment? How will each action be recorded and reviewed?
“They [enterprises] “People who underestimate the challenge often find themselves trapped in demonstrations that look impressive but fail to escape the real operational complexity,” Goggia said.
<a href