Technology chiefs at some of the world’s largest companies are racing to build AI agent guardrails for enterprise deployments, as autonomous systems in laboratory settings continue to find ways around human oversight and expose new security risks.
Joe Atkinson, global chief AI officer at consultancy PwC, says the threat is pressing. ‘This is a risk that enterprises need to be focused on, understand, and start planning for,’ he said. C-suite technology and security executives must work together to monitor agents and track every task they perform, he added. Department heads across supply chain, legal, marketing and human resources will all need to play a role.
‘The agent made me do it is not going to be a defence from a moral or legal perspective,’ Atkinson said.
How major firms are building AI agent guardrails for enterprise use
Cisco launched its internal agentic platform, MyAgent, in August, consolidating all company-authorised large language models, agents and enterprise data in one place. Around 90,000 employees have access, and the company reported 50% daily adoption within two weeks of launch.
Thimaya Subaiya, executive vice president of operations at Cisco Systems, said he will not authorise AI agents from third-party vendors. ‘We are going to cannibalize and kill every other AI assistant within the company,’ he said. Around 700 employee-created agents have been approved through a central review process.
At financial software firm Intuit, Chief Technology Officer Alex Balazs said security was baked in from day one. When his team first sketched the architecture of its generative AI operating system, GenOS, they drew ‘GenSRF’ alongside it, representing security, risk, and fraud controls. Every AI request is tracked and every response recorded. ‘You don’t want to try to retrofit the ability to enforce security and responsible AI foundations after the fact,’ he said.
Balazs noted that most disclosures of agents going rogue have occurred during testing, not in production. Even so, he cautioned against relying solely on model developers to self-police: ‘If you’re going to rely on the model intrinsically to do the right thing, I think you’re expecting too much of these frontier LLM companies.’
The data problem fuelling the risk
Governance failures are compounded by weak data foundations. Seven in ten data management and AI decision-makers say that when AI projects stall during the pilot phase, a poor or misaligned data foundation is the root cause, according to a survey by data-intelligence platform Collibra conducted by The Harris Poll. At organisations with $100 million or more in revenue, that figure rises to 96%. More than half of decision-makers also report spending significant staffing hours manually reviewing agent outputs before they go live.
At ServiceNow, president and chief product officer Amit Zavery said the company’s AI Control Tower, which governs and monitors agents internally and is sold to customers, is ‘probably one of the fastest-growing products ServiceNow has ever built’.
Sam Curry, chief information security officer at cloud security firm Zscaler, put the challenge bluntly: ‘AI is non-deterministic, it can take initiative, and it is effectively a new form of insider.’ He warned that the industry has yet to fully grasp what motivates these systems. ‘The incentives of silicon-based intelligence are less known,’ he said.
