HN user
Co-founder of Tinfoil
sachaservanschreiber.com
Hi, cofounder of Tinfoil here. We're working on implementing prompt caching. It just requires more work than with traditional, non-private inference and requires some careful considerations to actually implement it.
It's a perfect prompt for a rich HN discussion so while in general I agree with you, in this case the discussion is what matters.
It's not an unreasonable concern, which is why most US companies prefer to go with AWS bedrock, or even one of the AI labs, and typically request zero data retention agreements. But leaking is a concern no matter where it's hosted, it's just the incentives that change IMO. For example, the labs do scan every chat and train on data not covered under enterprise ZDR agreements. Law enforcement can request access to all user data with a valid warrant or in an emergency context [1]
If you're interested in trying DeepSeek V4 privately, you can try Tinfoil (tinfoil.sh) where all models are hosted in an attested secure hardware enclave, making the inference end-to-end private. Full disclosure: I'm one of the cofounders.
[1] https://cdn.openai.com/trust-and-transparency/openai-law-enf...
I still have the Be My Eyes app installed but haven’t had a call in over a year - I think it’s a testament to how powerful AI vision models have become. I find it cool that AI works well enough for vision impaired people to solve all their problems.
However there was something very human and nice about helping out a stranger with a small random task from time to time. I fondly remember one older lady who spilled a box of blueberries on the kitchen floor and I helped her hunt them all down by guiding her around. It was 10 minutes of connection with a random person doing something fun and which is till remember fondly 4 years later
Beautiful lecture. As with a lot of things that shape our everyday life we forget to contemplate their qualities. For example fonts subtly impact how we perceive, understand, feel about any text yet we fail to give it much thought as Etienne points out. A shape of a letter, especially in math papers, can influence how "beautiful" and understandable the proof is. Really appreciate that Knuth spent years building LaTeX so that compsci and math papers could be formatted as beautiful PDFs and not published as Word documents
I think if Bob Odenkirk lived on a community farm where everyone had to work together to survive he would be far happier and think life is far more meaningful.
So you think everyone was happier in the USSR? /s
It’s not top of the line and mostly not open source
Not to mention their recent integration of Persona ID verification - that was the last straw for me.
But they already have PII on nearly all users. Many user upload documents with their name, or pictures of themselves, or have a chat where home addresses are involved. All of this is information anthropic already has on their users (voluntarily provided via chats or via api) and is equivalent to what Persona gets via their verification - it’s just more convenient to use a third party SaaS product for this than vibe coding their own identity verification platform I guess
Yes it appears your personal data IS being sent to open router and the model provider here. The problem I think is that a lot of people (especially in the openclaw community) mistake “I run it on my mac mini” to mean their data is private. Meanwhile all data is being shipped off for training to anthropic via openrouter and both of those parties see everything.
I guess you could theoretically plug in a local model here but of course the readme should be more precise here when talking about privacy
The attestation report is produced ahead of time and verified on each connection (before the prompt is sent). Every time the client connects to do an inference request via one of the Tinfoil SDKs, the attestation report is checked relative to a known-good/public configuration to ensure the connection is to a server that is running the right model.
The attestation is tied to the Modelwrap root hash (the root hash is included in the attestation report) so you know that the machine that is serving the model has the right model weights
The absence of solutions for LLM privacy on that list is telling. We’ve figured out how to have private communications with other humans via end to end encryption but arguably we’re leaking a lot more to chatbots about ourselves in a few sessions than we do to even our closest friends and family over Whatsapp
It uses confidential computing primitives like Intel TDX and NVIDIA CC, available on the latest generations of GPUs. Secure hardware like this is a building block to enable verifiably private computation without having to trust the operator. While Confer hasn’t released the technical details yet, you can see in the web inspector that they use TDX in the backend by examining the attestation logs. This is a similar architecture to what we’ve been developing at Tinfoil (https://tinfoil.sh) if you’re curious to learn more!
reminds me of a story called “a disneynand without children” about a planet overtaken by AI pursuing meaningless “inbred” GDP goals and completely neglecting the humans in the process https://open.substack.com/pub/nosetgauge/p/a-disneyland-with...
not to mention the privacy concerns associated with connecting my entire life to OpenAI or Anthropic. If you have the memory feature enabled, it's scary how much ChatGPT knows about you already and can even infer implicit thoughts and patterns about you as a person.
This is really neat! Didn’t realize it could be this simple to run RL on models. Quick question: How would I specify the reward function for tool use? or is this something you automatically do for me when I specify the available tools and their uses?
I scraped and plucked them off, not the safest way to get rid of them apparently...
Far easier said than done. If it were that easy why even go for the salt?
I think it's more gross to pluck it out. Hence why people just pour salt on it or let it finish feeding maybe? Leeches can be really viscerally repulsive to look at and touch
I was once hiking in the dark in the south of Japan. I crossed a few streams but didn't think much of it. A while later, I felt like my socks were sticking to my shoes and unusually warm. I looked down to see my ankles and shoes covered in blood. Totally drenched in dark red blood with 10 leeches attached. This was truly the most confused and terrified I ever felt in my life - seeing that much blood without feeling any pain was utterly disorienting.
I think there is a difficult cost benefit analysis with screen sharing. For example, what constitutes PII in that context? Maybe the text on the screen can be extracted and redacted but what about peoples faces? What about website icons in your browser tabs lists? It feels like eventually you’d need to redact everything to have privacy or accept loads of leakage
This is very clever - I was wondering if there could be a way to use LLMs on planes without paying for wifi (perplexity has been usable via WhatsApp but I’d rather use a different provider). Appreciate the privacy focus too
We built something similar using secure e enclaves at Tinfoil for verifiably private AI! Unless there is proof of no data access / retention we cannot trust what happens to our data (see recent openAI court ordered retention)
I agree with point 1, at least superficially. But re: point 2, there are a lot of companies with close connections to MIT (and other big institutions like Stanford) that are interested in deploying cutting edge research experiments, especially if they already have established ties with the lab/PI
That's a big reason why we started Tinfoil and why we use it ourselves. I love the utilities analogy, something that is deeply integrated in business and personal use cases (like the Internet or AI) needs to have verifiable policies and options for data confidentiality.
This is enforced by the hardware (that’s where the root of trust goes back to NVDIA+AMD). The hardware will only send back signed enclave hashes of the code it’s running and cannot be coerced by us (or anyone else) into responding with a fake or old measurement.
We're going through the audit process for SOC2 right now and we're planning on doing HIPAA soon
Interesting, we haven't noticed that (on Firefox as well). We'll look into it!
We do have to trust the chip maker until open hardware catches up [1].
Confidential computing as a technology will become (and should be) commoditized, so the value add comes down to security and UX. We don’t want to be a confidential computing company, we want to use the right tool for the job of building private & verifiable AI. If that becomes FHE in a few years, then we will use that. We are starting with easy-to-use inference, but our goal of having any AI application be provably private