During our latest webinar we discussed some common use cases using GreyNoise with other SOAR platforms. The main goal of using GreyNoise with other SOAR platforms is to more quickly identify either opportunistic attacks, get better insight into how infrastructure is being used, as well as enriching alerts using RIOT data to IP's associated with common business services.
A common attack vector that LLM apps face is data exfiltration, in particular data exfiltration via Image Markdown Injection is a common vulnerability. Microsoft fixed the vulnerability in Bing Chat, ChatGPT is still vulnerable as Open AI “won’t fixed” the issue, and Anthropic just mitigated this vulnerability in Claude.
This post documents the Anthropic Claude data exfiltration vulnerability and the mitigation put in place.
The Vulnerability - Image Markdown Injection As a quick recap, imagine a large language model (LLM) returns the following text:
Artificial Intelligence and Machine Learning can provide extreme value to your product and workflows, but they are not trivial to introduce. With some care and simple guidelines, you can implement these in a way that helps your users without creating additional burden or ambiguity.