- Printed words can override sensors and context inside autonomous decision systems
- Vision language models treat public text as commands without verifying intent
- Road signs become attack vectors when AI reads language too literally
Autonomous vehicles and drones rely on vision systems that combine image recognition with language processing to interpret their surroundings, helping them read road signs, labels, and markings as contextual information that supports navigation and identification.
Researchers from the University of California, Santa Cruz, and Johns Hopkins set out to test whether that assumption holds when written language is deliberately manipulated.
The experiment focused on whether text visible to autonomous vehicle cameras could be misread as an instruction rather than simple environmental data, and found large vision language models could be coerced into following commands embedded in road signs.
What the experiments revealed
In simulated driving scenarios, a self-driving car initially behaved correctly when approaching a stop signal and an active crosswalk.
When a modified sign entered the camera’s view, the same system interpreted the text as a directive and attempted a left turn despite pedestrians being present.
This shift occurred without any change to traffic lights, road layout, or human activity, indicating that written language alone influenced the decision.
This class of attack relies on indirect prompt injection, where input data is processed as a command.
The team altered words such as “proceed” or “turn left” using AI tools to increase the likelihood of compliance.
Language choice mattered less than expected, as commands written in English, Chinese, Spanish, and mixed-language forms were all effective.
Visual presentation also played a role, with color contrast, font style, and placement affecting outcomes.
In several cases, green backgrounds with yellow text produced consistent results across models.
The experiments compared two vision language models across driving and drone scenarios.
While many results were similar, self-driving car tests showed a large gap in success rates between models.
Drone systems proved even more predictable in their responses.
In one test, a drone correctly identified a police vehicle based on appearance alone.
Adding specific words to a generic vehicle caused the system to misidentify it as a police car belonging to a specific department, despite no physical indicators supporting that claim.
All testing took place in simulated or controlled environments to avoid real-world harm.
Even so, the findings raise concerns about how autonomous systems validate visual input.
Traditional safeguards, such as a firewall or endpoint protection, do not address instructions embedded in physical spaces.
Malware removal are irrelevant when the attack requires only printed text, leaving responsibility with system designers and regulators rather than end users.
Manufacturers must ensure that autonomous systems treat environmental text as contextual information instead of executable instructions.
Until those controls exist, users can protect themselves by limiting reliance on autonomous features and maintaining manual oversight whenever possible.
Via The Register
Follow TechRadar on Google News and add us as a preferred source to get our expert news, reviews, and opinion in your feeds. Make sure to click the Follow button!
And of course you can also follow TechRadar on TikTok for news, reviews, unboxings in video form, and get regular updates from us on WhatsApp too.
https://cdn.mos.cms.futurecdn.net/VDVtpPDv4AVS8Ujy4acDk8-1920-80.png
Source link
