As global AI spending tops $2.5 trillion this year, many companies still aren’t seeing meaningful returns. With rising pressure to justify investments, they’re betting on AI agents to right the ship. But if agents are going to deliver the value companies are waiting for, alignment with human judgment can’t be an afterthought.
CONTAINMENT VERSUS ALIGNMENT
As companies stand up their AI governance programs, they often begin with inventories, security guardrails, access policies, and monitoring. I call this containment. Think of it as the brakes in a self-driving car. It is the programming that allows systems to respond to stop signs, traffic lights, and other formal rules of the road. Containment tells the system what it can’t do.
But AI agents are forcing businesses to confront a more existential challenge: embedding human judgment into autonomous systems that make decisions at AI speed. How do we design AI to operate within an organization’s values, policies, risk tolerance, and understanding of context as conditions change?
This is alignment. Alignment helps determine what the system should do when the right answer depends on context. While guardrails can stop an agent from crossing a line, they don’t tell an agent how to exercise judgment when no line is clearly marked. Think of it as the self-driving car’s ability to read context and yield to a funeral procession, even when there is no law in place.
This includes compliance with policies, data use rules, and ethical guardrails, but it also means anchoring agents to the actual business outcomes the organization is trying to drive. An agent that follows every rule while drifting from the company’s strategic priorities and brand promise is still misaligned.
Employees often apply this judgment as second nature. We observe behaviors over time, we recognize regional and cultural nuances, we challenge an idea that looks good on paper but fails in the real world, and we understand when methods compromise the outcome.
But AI agents don’t.
THE RISK OF CONTINUOUS OPTIMIZATION
There’s an example I use to explain the need for alignment. A meal subscription service has a marketing agent designed to optimize campaign performance. With a set budget and goals, the agent accesses datasets, analyzes chat and support logs, identifies customer segments, and delivers promotions. By the time the campaign ends, the agent has reached its sales and profitability goals.
But, behind the scenes, something else happened. The agent delivered aggressive advertising with higher pricing disguised as “limited-time discounts” to people who previously mentioned financial stress or health concerns during support calls and chats.
When the incident becomes public, the fallout is extensive. Price gouging—especially targeted at the company’s most vulnerable clients—violated ethical use policy and directly conflicted with the company’s mission statement and values. As a result, customers cancel their subscriptions en masse, regulators begin an investigation, and any revenue gained through the initial campaign is lost.
This story shows just how quickly an agent can cause issues without ever technically malfunctioning. Exploiting certain customers wasn’t part of the prompt; it was just the pattern that improved results. Discrimination, privacy issues, and policy violations can occur regardless of the company’s intention.
Ultimately, that’s because agents are systems trained to maximize efficiency. Continuous optimization enables them to complete their objectives.
It’s also the reason why agents need alignment. Alongside security and access guardrails, policies ensure agents optimize only within the limits set by the business.
WE’RE AT AN INFLECTION POINT
Gartner predicts that large enterprises will have over 150,000 agents in use by 2028, up from over a dozen per company today. And that number is quickly rising thanks to trends like tokenmaxxing and corporate incentives to leverage AI.
Now, the challenge becomes how to encode agents with human judgment at scale. Traditional manual review processes were built for slower, more static systems, where teams had time to inspect, catch, and resolve issues before production. Unfortunately, no amount of hiring can help you keep up with hundreds and ultimately tens of thousands of agents optimizing at AI speed.
The good news is that there are fewer AI agents today than there ever will be. Now is the time to build automated governance, catalog your agents, define your baseline policies, and enforce them alongside security controls and guardrails. It is much easier to scale a program as your agent workforce grows than to try and retrofit one later.
Building AI for speed alone is short-sighted. Our goal should be building AI that moves fast—in the right direction.
Blake Brannon is chief innovation officer of OneTrust.