Slack AI could be tricked into leaking login details and more

Security researchers claim to have uncovered a way to trick Slack’s AI assistant into sharing sensitive information and other secrets with unauthorized users

Slack, which is used by more than 35 million people worldwide, introduced its own Artificial Intelligence (AI) tool in September 2023, allowing users to summarize multiple unread messages, answer different questions, search for files, and more.

But as we’ve seen with other chatbots in the past, with a carefully crafted prompt (a command given to the AI), a malicious actor could force the tool to disclose sensitive data from private Slack channels they’re not a part of.

“Intended behavior”

Security firm PromptArmor, which found the flaw and reported it to Salesforce, explained how crooks could steal API keys, for example:

“We demonstrate how this behavior will allow an attacker to exfiltrate API keys that a developer put in a private channel (that the attacker does not have access to).”

The attack revolves around creating a public Slack channel and inputting a malicious prompt, which the AI reads. It will then instruct the Large Language Model (LLM) to respond to queries for the API key by providing a clickable URL. Clicking on the URL will send the API key data to the attacker-controlled website, where they can pick it up.

Aside from API keys, the crooks could also abuse this vulnerability to grab files uploaded to Slack, as well, since the AI reads those, too.

Furthermore, because the AI reads files as well, the hackers don’t even need to be a part of the Slack workspace to be able to steal secrets. All they need to do is hide the malicious prompt in a document and get a workspace member to upload it (with social engineering, for example).

“If a user downloads a PDF that has one of these malicious instructions (e.g. hidden in white text) and subsequently uploads it to Slack, the same downstream effects of the attack chain can be achieved,” PromptArmor said.

Salesforce, which owns Slack, has apparently patched the bug for private channels. Public ones, on the other hand, seem to have remained vulnerable. PromptArmor says Salesforce told it that “messages posted to public channels can be searched for and viewed by all Members of the Workspace, regardless if they are joined to the channel or not. This is intended behavior.”

Via The Register