Zero-trust AI APIs serving Llama 2 70B inside enclaves

https://blindllama.mithrilsecurity.io/en/latest/

by DanyWin • 3 years ago

35 8 3 years ago

blindllama.mithrilsecurity.io

purplecats2• 3 years ago

brilliant! very excited to use this.

what will pricing look like (tried CMF-F)?

purplecats2• 3 years ago

also can you make your whitepaper accessible without the login/email hoops

mithril-docs2• 3 years ago

Hi, We've added a copy directly in our GitHub repo:

https://github.com/mithril-security/blind_llama/blob/main/do...

mithril-docs2• 3 years ago

Hi there,

Thanks for your question and sorry for the delay in getting back to you!

The pricing information is available here: https://www.mithrilsecurity.io/pricing

kromem• 3 years ago

I've been thinking about the privacy model of E2E encrypted channels with AI processing before sending back for a while now.

Awesome to see people doing it and wish you the best of luck.

jerpint• 3 years ago

How “secure” is this zero-trust? Are there cryptographic guarantees?

amluto• 3 years ago

This seems likely to be extremely weak. The TPM is somewhat useful for verifying that hardware that you own or fairly tightly specify is running what you think it is unless there is a moderately sophisticated physical attacker around. But for servers in the cloud? How are you even supposed to know what firmware, etc you’re verifying?

Actual confidential computing systems (TDX, fancy variants of SEV, etc) are meant to address this type of use case. The TPM isn’t.

vvoruganti• 3 years ago

Is the model being served using confidential GPUs?