Imagine the world’s most capable intern. Someone who can read thousands of documents overnight, make inferences from complex problems instantly, and work 24/7 without complaints. But there’s a catch: this intern is also incredibly gullible and will believe almost anything you tell them, making them the perfect target for manipulation by bad actors.
This analogy perfectly illustrates the current state of agentic AI. It’s simultaneously the most sophisticated tool ever created and the most vulnerable to simple deception.
This is made more challenging by how differently people view AI. The features that excite some terrify others, creating a divide between the builders and users.
Builders – aka engineers and researchers – focus on foundational challenges like data quality, algorithmic bias, and existential risks. Their concerns dominate headlines and academic discussions.
But users, such as business leaders and operational teams who want to harness these tools practically and safely, have more practical worries. Less concerned about whether AI will end the world and more focused on whether it will expose customer data or make costly mistakes.
While builders are focused on the future, users want to know what AI can do today. And unfortunately, the gap between what they expect agentic AI to deliver and what it can is substantial.
Dr Damien Duff is Principal AI/ML Consultant at Daemon.
Expectation vs reality
The narrative around agentic AI often paints a picture of fully autonomous digital workers able to alter a business overnight. While multi-agent LLMs are no longer theory, there’s more exploration that must be done before they can enable complete business transformation.
Current AI systems can deliver impressive agent-like behaviors including knowledge extraction from vast documents, accelerating the software delivery lifecycle, and empathetic customer interactions. But truly autonomous systems that work independently in complex novel environments remain out of reach.
While AI can complete structured tasks with human oversight, it struggles with open-ended problems, long-term planning and high-stakes decision making where failures have consequences.
For example, AI can identify potential vulnerabilities in code and propose wide ranging fixes, but developers must evaluate the solutions to implement and guide their application as AI can’t consistently decipher broader system context.
The danger of inflated expectations is that organizations are caught off guard by the real security risks. Lured into a false sense of readiness, they’re ill-prepared for the routine threats they’re far more likely to face.
The security challenge
As agentic AI systems become more autonomous, organizations face a critical challenge of ensuring systems act aligned with business goals. As agentic AI becomes more capable, it also becomes harder to control and therefore, easier to exploit. Unlike traditional software, which fails in predictable ways, AI systems can fail creatively, manipulated in ways their creators never anticipated.
So, what kinds of security risks should organizations actually be worried about?
While traditional IT challenges, such as data protection across systems, risk management, robust reporting and visibility remain critical, there are other novel challenges that require fresh approaches.
For example, adversarial prompt engineering, such as prompt injection where bad actors can embed malicious instructions in innocent requests, or context manipulation, where attackers provide false context, causing AI to make decisions based on incorrect assumptions.
Another significant issue is accumulation of errors: while experienced humans often spot their mistakes, AI errors can quickly snowball, especially in multi-agent systems, turning small problems into big ones; this can go unnoticed in the early stages as agentic reasoning is superficially similar to human reasoning, and then quickly spiral.
AI risks may seem daunting, but the solutions are often more familiar than organizations expect. It’s not a question of if a security problem will arise, but a matter of when, so it’s important to be prepared.
The solution
Many of the solutions for agentic AI’s specific security challenges exist in traditional cybersecurity and risk management frameworks. The approach requires companies to apply principles they’re already familiar with (or with which organizations they work with are familiar), including zero trust, human oversight, and controlled access.
Rather than assuming all AI inputs are safe, companies should treat them as potentially malicious and implement multiple validation layers. This approach works across all applications, whether customer service or financial operations.
While the principles are straightforward, successful implementation requires careful planning. To build effective agentic AI security, organizations should:
- Start with a small blast radius: Begin with low-risk, high-value use cases where mistakes are recoverable. For example, deploy AI for document summarization before moving onto financial transactions. This builds organizational confidence and expertise.
- Build governance: Don’t wait for problems to emerge. Establish clear approval processes and staff training programs so employees know what to do when things go wrong.
- Automate the validation process: Build and continually grow comprehensive AI-based and traditional test suites that encapsulate the worst imagined case scenarios of adversarial attacks, edge cases, and so forth, so that we are not following a moving target but measuring our improvement carefully and preventing backslides.
- Question everything: Train users and teams integrating AI tools to question AI outputs rather than inherently trusting them. Make verification part of company culture amongst both builders and consumers by setting up peer review processes, introducing spot-checking protocols and celebrating when someone catches a mistake.
- Develop protection processes: Conduct regular risk assessments, audit existing implementations for vulnerabilities, implement monitoring systems, and create AI-specific incident response procedures. Set up security policies for success.
- Stay ahead: Ensure employees are informed about emerging threats by providing relevant training, as well as encouraging them to participate in industry forums, discuss experiences with peers, and attend security conferences. Remember, knowledge is power.
The future
Agentic AI is a powerful tool that can transform how we work and solve problems. But like any powerful and emerging technology, it requires respect, understanding, and proper safety measures.
The key is approaching this technology with the same careful planning and risk management we should apply to any other tool. The gullible savant intern analogy reminds us that even the most capable systems need supervision, clear boundaries, and ongoing guidance.
We’ve featured the best AI website builder.
This article was produced as part of TechRadarPro’s Expert Insights channel where we feature the best and brightest minds in the technology industry today. The views expressed here are those of the author and are not necessarily those of TechRadarPro or Future plc. If you are interested in contributing find out more here: https://www.techradar.com/news/submit-your-story-to-techradar-pro
https://cdn.mos.cms.futurecdn.net/cJwCzahwRcWWKn8F8NYm25.png
Source link
