In the past year, the enterprise AI ecosystem has gained enormous capability and zero consensus.
Developers now have a remarkable set of tools for building AI agents: OpenAI’s frameworks, Anthropic’s Claude tooling, LangChain, LangGraph, CrewAI, Microsoft AutoGen, and a growing list of alternatives. Each promises to coordinate reasoning loops, manage multi-step task execution, and connect agents to tools and APIs. For experimentation, the progress has been substantial. Teams can now assemble sophisticated agent workflows in days that would have taken months two years ago.
But I’ve watched this pattern before. In over two decades of building and selling distributed systems platforms, I’ve seen the same dynamic play out across nearly every major infrastructure shift: the tools for consuming a new capability arrive before the infrastructure for governing it does. The gap that emerges isn’t immediately obvious in development environments. It becomes obvious in production.
That’s exactly where enterprise AI stands today.
What agent frameworks don’t handle
Modern agent frameworks are fundamentally coordination systems. They determine what a system should do: which tools to call, how to sequence tasks, how to delegate work across agents. That’s hard work, and they’ve gotten quite good at it.
What they rarely address is where those tasks are allowed to run, and under what conditions.
Take a seemingly simple workflow: summarize customer support transcripts using an LLM. In a development environment, the implementation is clean. The agent calls a model API, passes the transcript, and returns a summary. In production at an enterprise, the same request may involve a dataset that can’t cross a specific geographic boundary, a model that isn’t approved for regulated data, and an audit requirement that demands a traceable record of what happened.
Those aren’t planning problems the agent framework was designed to solve. They’re execution governance problems. Most frameworks quietly assume they’re handled somewhere else in the stack. In many enterprise environments, they’re not handled at all. Gartner predicts more than 40% of agentic AI projects will be canceled by the end of 2027, citing inadequate risk controls as a primary driver of failure—a number that reflects exactly this gap.
What the missing layer actually does
Addressing these governance problems requires an additional layer between agent logic and execution: one that evaluates every agent action against policies governing where data can reside, which models may process it, who authorized the request, and how the action fits within the organizational context. The agent framework determines what the system should do. The orchestration layer determines whether and where it’s allowed to happen. Keeping those responsibilities separate allows both layers to evolve independently. It also means you can adopt new agent frameworks without rebuilding your governance model from scratch.
This separation will feel familiar to anyone who has worked through the Kubernetes era. Kubernetes doesn’t care what’s inside your container. It finds capacity, allocates resources, and ensures things run. The orchestration layer for agentic AI plays an analogous role: it doesn’t care which agent framework generated the request. It enforces the conditions under which that request can execute.
Richer authorization models
Traditional enterprise access control is built around a simple question: can user X access resource Y? That’s insufficient for autonomous agents.
A realistic authorization decision for an agent request might look more like this:
request = {
"agent": "support-summary-agent",
"task": "summarize",
"dataset": "customer_support_logs",
"model": "external_llm_api",
"delegated_by": "user_4821"
}
policy = evaluate_policy(request)
if policy.allowed:
route_to_execution(policy.execution_environment)
else:
raise AuthorizationError(policy.reason)
The policy engine here evaluates dataset classification, model approval status, geographic processing rules, and the delegation chain that initiated the request. That might mean redirecting the task to an internal inference cluster instead of a public API endpoint, or blocking the request if no compliant execution environment exists. From the agent’s perspective, the task still executes. The orchestration layer ensures it runs in an environment that satisfies enterprise policy.
Why ontologies are load-bearing infrastructure
For the orchestration layer to make good decisions, it needs to do more than label data. It needs to understand how the entities involved in a request relate to each other, and reason over those relationships to determine what’s allowed.
Consider the customer support transcript example again. Metadata tells you the dataset contains PII (personally identifiable information). An ontology lets the system reason across a connected chain: the task operates on a dataset containing personal data; that data is governed by GDPR; the organization’s policy requires processing within an approved EU environment; the selected model runs outside that boundary. From those four connected facts, the orchestration layer can infer the request must be rerouted or blocked. The system reasoned over the relationships rather than matching against a hardcoded rule tied to a specific dataset.
This is what makes policy enforcement, execution routing, data locality, and audit decisions computable at runtime. An ontology can be built around virtually any entity-relationship set the enterprise needs to govern: datasets, models, agents, users, regulations, tasks, environments. The relationships that matter are the ones that drive the decisions the governance layer needs to make. Access control lists can restrict who touches a resource, but they can’t reason across a connected set of entities. That reasoning is what the orchestration layer depends on.
Decision provenance as a first-class requirement
Enterprise systems also require auditability. When automated agents trigger actions across multiple systems, organizations must be able to reconstruct the decision path that produced the outcome. Compliance depends on it. So does incident response and basic operational trust.
An orchestration layer generates records describing the initiating identity, the agent, the model, the data sources, the policies evaluated during authorization, and virtually anything else the organization chooses to capture in its ontology. That chain of custody allows teams to investigate incidents and validate compliance without treating production AI systems as operational black boxes.
Regulators and auditors are no longer satisfied with knowing what an AI system was designed to do. They want a factual record of what it did in a specific instance, under what authorization, and with what effect—something dashboards can’t provide, but a well-designed orchestration layer can. The EU AI Act makes this explicit: under Article 12 and Article 17, high-risk AI systems must maintain documentation that makes decisions traceable and auditable, with records sufficient to support investigation after the fact.
Where this leaves enterprise teams
Agent frameworks will keep improving. The coordination problems they solve are real, and the ecosystem will continue to mature. But the architectural challenge for enterprises has shifted. It’s no longer primarily about coordinating agents. It’s about governing how those agents interact with real infrastructure, real data, and real compliance obligations.
The patterns for doing that exist today: contextual authorization, data locality enforcement, ontology-aware policy evaluation, decision provenance. What most organizations are missing is the recognition that these capabilities belong in a distinct layer that operates independently of whichever agent framework sits above it. Build that layer, and the rest becomes manageable.
—
New Tech Forum provides a venue for technology leaders—including vendors and other outside contributors—to explore and discuss emerging enterprise technology in unprecedented depth and breadth. The selection is subjective, based on our pick of the technologies we believe to be important and of greatest interest to InfoWorld readers. InfoWorld does not accept marketing collateral for publication and reserves the right to edit all contributed content. Send all inquiries to [email protected].