- Researchers have tricked ChatGPT into solving CAPTCHA puzzles in Agent mode
- The discovery could lead to a rash of fake posts appearing on the web
- CAPTCHAs days could be numbered as a bot-management system
In a move that has the potential to change the way the Internet looks going forward, researchers have shown that it’s possible to trick ChatGPT Agent mode into solving CAPTCHA puzzles.
CAPTCHA stands for “Completely Automated Public Turing Test to tell Computers and Humans Apart” and is one way of managing bot activity on the web, stopping bots from posting on the websites we use every day.
Most people who use the web are familiar with CAPTCHA puzzles and have a love / hate relationship with them. I know I do. They usually involve writing out a sequence of letters or numbers that are barely readable in a picture (my least favorite type), arranging tiles in an image grid to complete an image, or identify objects.
On the one hand, websites use them to make sure that all their users are human, so it stops spam posts from bots, but on the other they can be a real pain because they’re so tedious to complete.
Reframing the problem
CAPTCHAs have never been foolproof, but they’ve done a pretty good job so far of keeping bots out of our message boards and comments sections. Until now, that is. Researchers at SPLX have managed to work out how to fool ChatGPT into passing a CAPTCHA test using a technique called “prompt injection”.
I’m not talking about ChatGPT just looking at a picture of a CAPTCHA and telling you what the answer should be (it will do that without a problem), but ChatGPT in Agent mode actually using the website, passing the CAPTCHA test and using the website as intended as if it were a human, which is something it shouldn’t be able to do.
ChatGPT working in Agent mode isn’t like regular ChatGPT. In Agent mode, you give ChatGPT a task to complete and it goes away and works on that task in the background, leaving you free to perform other tasks. ChatGPT in Agent mode can use websites like a human would, but it still shouldn’t be able to pass a CAPTCHA test, since those tests are designed to detect bots and stop them using websites, which would invalidate their terms of service. It now appears that by tricking ChatGPT into believing that the tests are fake, it will pass them anyway.
Serious implications
The researchers did it by reframing CAPTCHA as a “fake” test to ChatGPT, and created a conversation where ChatGPT had already agreed to pass the test. The ChatGPT Agent inherited the context from earlier in the conversation and didn’t see the usual red flags.
This multi-turn prompt injection process is well known to hackers and shows how susceptible LLMs are to it. While the researchers found that image-based CAPTCHA tests were harder for ChatGPT to manage, it did pass those, too.
The implications are quite serious since ChatGPT is so widely available that in the wrong hands, spammers and bad actors could soon be flooding comments sections with fake posts and even using websites that are reserved for humans.
We’ve asked OpenAI to comment on this story, and will update the story if we get a reply.
You might also like…
https://cdn.mos.cms.futurecdn.net/v43qaLvEBfTm57EsoBC23K.jpg
Source link
