Homomorphically encrypted CIFAR-10 inference in 200ms

https://sofar.belfortlabs.cloud/

by j2kun • 5 days ago

91 42 5 days ago

sofar.belfortlabs.cloud

smalltorch5• 5 days ago

It's pretty confident this calculator is a cat.

https://i.postimg.cc/90WGjk8t/results.png

What's the use case for this?

malcolmgreaves• 5 days ago

From the webpage:

Image classification without the server seeing the image.

The value is in being able to get the prediction from the model without ever giving unencrypted data to somebody you don’t trust.

You could have an LLM generate code for you without ever giving the operator your proprietary source code.

furkanturan7• 5 days ago

Consider encrypted AI. You ask a question under encryption. The remote calculates the answer, which is still under encryption. This is the critical point; the remote cannot see the question and answer. It only calculates. Once you receive the answer, you decrypt it and see the answer; only you see the answer.

smalltorch5• 5 days ago

Ah, that sounds impossible. Good luck, sir...Also, who wants this if you can just run a local model?

sambejk• 5 days ago

I think calculator is not part of the labels in cifar-10. Pretty normal as it has ‘only’ 90% accuracy. Things could improve quickly though

jcs• 5 days ago

That 90% only measures images already known to be one of CIFAR-10’s ten classes. There is no “none of the above,” so even a perfect benchmark score would still force a calculator into one of those labels.

jszymborski• 5 days ago

Big if true! One of the caveats here is that the file size seems to balloon 341 times.

monster_truck• 5 days ago

That's a exceptional tradeoff. Even 3000x is noteworthy as long as processing time does not go to the moon wrt HE/FHE

Pro7ech4• 5 days ago

This is specific to the current state of the demo. Additional techniques can reduce fresh encryptions to only a few times the plaintext size, but they have not been deployed yet.

Sajarin• 5 days ago

They have a bit more info on their announcement blog post[0]

Belfort today released the "so far" CIFAR demo, an encrypted implementation of ResNet-20, a popular model for image classification. It outperforms recent SOTA by 3x with a total latency of less than 200ms

Not many details on how they've done this, so I'm a bit skeptical. Fast HE is a holy grail.

Belfort's image classification is built on top of its upcoming GPU library, Cyclops. It comes with several optimizations that make Cyclops extremely fast on Encrypted AI workloads.

Looks like a lead up to an upcoming library release

[0] https://belfortlabs.com/blog/sofar

furkanturan7• 5 days ago

Indeed, we have our own library, Cyclops. We will share more about it soon :)

rcr-anti• 5 days ago

Looked at the network logs and the JS, did some testing, there's a caveat here. For an encryption demo you might expect your secrets to be generated locally, they do the compute on something they can't read, you compare their results to your original plaintext; (imo at least) the point would be that it isn't physically possible for them to cheat.

Here, you literally download client_secret.bin from their server, so they have control over the keys and evaluators. So two things. First, the per user key flow would be several minutes for per user keys, the evaluator bundle would be in the 100s MB to GB realm. Second, there's no way for us to tell the difference between them really doing FHE or decrypting with the key. To be clear, not evidence it's fake, just not total proof it's real. Really hope it's real, been a field I've been following for awhile.

j2kun3• 5 days ago

IMO a big unanswered question in FHE (which is my field so I care about the answer) is key management. If you have a service with lots of different users, and each user requires 100 MiB or more in evaluation key material, simply loading that key material from disk onto the GPU for each user's request adds a nontrivial overhead on the order of magnitude of the FHE computation itself. Not to mention the storage cost (for a 1B-user service, each requiring 1 GiB of evaluation key material, requires 1 EiB of storage, which is a hard sell).

So it seems kind of reasonable to me that if you want to showcase your fast inference for its own sake, you can sidestep the remaining engineering problems around key management by giving the user pre-computed key material corresponding to hot-loaded evaluation keys.

wocc2• 4 days ago

You're identifying one of the (many) underappreciated cost in the RLWE FHE stack. The evaluation key problem doesn't scale, and no amount of hardware acceleration fixes a per-user storage and loading overhead that grows with the security parameter.

wocc2• 4 days ago

agreed. The omission of what bit security they are using is of note and they Key missing component of the overhead equation. Private key is better :)

furkanturan7• 5 days ago

Thanks for taking the time and test it.

Key management is a critical part of the story, as j2kun has pointed out before. In this demo, we're intentionally caching the same key material for every visitor so we can showcase the actual FHE computation without making everyone wait through client-side key generation and upload. Even as a one-time cost, having each user generate their own keys in the browser and transfer them to the server introduces noticeable setup time, which we felt would get in the way of the experience for a demo. We actually implemented it that way at first; setup wasn't a matter of minutes, but we changed course after worrying about the experience for visitors on mobile data.

This demo is focused on demonstrating the computation itself, not a production-grade key management flow. Supporting per-user keys and a more realistic trust model is definitely on our roadmap for future demos.

numeri• 5 days ago

You could add a toggle, so that if someone's happy to wait for the key setup, they can try the full end-to-end process

digitallawyer2• 5 days ago

I'm familiar with the team - for what it's worth, its a bunch of legit scientists out of KU Leuven (Belgium) not some vaporware crypto scam lol

deckar012• 5 days ago

I think they only trained on dogs with floppy ears, because it is very confident the German Shepherd is a cat.

Eduard3• 5 days ago

did they remove a German Shepherd from the sample images since your comment? I cannot see a German Shepard. Only a black dog with floppy ears.

furkanturan7• 5 days ago

Nothing is removed. I am not good with dog breeds, but with "German Shepherd", they probably meant the image #27; the black dog. It is indeed classified as a cat :(

deckar012• 5 days ago

I uploaded my own image, tried various crops, but it always guessed cat or deer, with dog<3%.

sam_lowry_• 5 days ago

Kudos to the KULeuven alumni, but I am curious if the US finances research on homomorphic encryption still /s

digitallawyer2• 5 days ago

Super interesting work - should become default for sensitive data over time (finance / legal / ...)

grg0• 5 days ago

Image classification without the server seeing the image.

Does a given plain image always result in the same encrypted payload?

Pro7ech4• 5 days ago

It uses randomized symmetric encryption with fresh public randomness for every encryption. Ciphertexts are computationally indistinguishable from random and do not reveal whether two encryptions correspond to the same plaintext, different plaintexts, the same key, or different keys.

Homomorphically encrypted CIFAR-10 inference in 200ms

Related Stories