VectorSmuggle: Covertly Exfiltrate Data in Embeddings

https://github.com/jaschadub/VectorSmuggle

by smugglereal • 1 year ago

36 6 1 year ago

A comprehensive proof-of-concept demonstrating sophisticated vector-based data exfiltration techniques in AI/ML environments. This educational security research project illustrates potential risks in RAG systems and provides tools for defensive analysis.

acmiyaguchi• 1 year ago

The idea of using stenographic techniques to exfiltrate data is interesting, but I don't quite follow the general method outlined in the repository -- either through the generated documentation or code. The threat model and case studies seem contrived. I find it hard to believe that folks would expose data via RAG that they wouldn't want users of the underlying system to be privy to.

There's too much fluff here to be useful. I imagine having something that is concise and concrete would make it more appealing to others. But as-is, it's missing a good technical summary and demonstration.

smugglereal2• 1 year ago

Thanks for the feedback!

It's less about the RAG exposing new data to a regular user, and more about using the vector pipeline as a covert channel. The idea is to sneak out data the attacker already can access, but in a way that might bypass traditional DLP looking at emails, USBs, etc.

The "fluff" is largely educational material, as the project is for research and learning. For a concrete technical demonstration, the scripts/embed.py and scripts/query.py scripts are the core, and the docs/guides/quick_start.md tries to offer a direct path to seeing it in action.

Hope that helps! Will add a video demo soon.

anonymousiam• 1 year ago

Well over a decade ago, I recall learning about a covert data exfiltration method that could bypass firewalls by using DNS lookups. The payload would be a base64 hostname prefix attached to an evil domain. Adding a time stamp to the prefix data would guarantee uniqueness, and get around local caching DNS servers.

DrScientist• 1 year ago

Yep - bottom line you just use a protocol you know the firewall won't/can't block.

In theory you don't even need anything in the payload - you could put information in the timing of the DNS requests a la morse code....

HTTP is the obvious other one - with much more options for somebody to exfiltrate data - you can think of ways where you don't even need an evil domain.

For example - you could exfilrate data via hackernews comments!

As far as I can see, the only thing you can do in the end is to make it harder to do easily, and then monitor unusual activity - and hope that is enough to stop large scale exfiltration, as small scale is impossible to stop.

stephantul• 1 year ago

Literal attack vectors

VectorSmuggle: Covertly Exfiltrate Data in Embeddings

Related Stories