tygra

Yo! You’re a serious guy, Jonah!

I run Qwen3 locally for coding and writing. It’s a solid model.

Framework Desktop is becoming really popular. The one with the Max+ 395 and 128GB of RAM is an absolute beast. I might buy a Beelink GTR9 Pro (Max+ 395 with 128GB RAM), which costs around $2,000.

llama.cpp is the real deal. I’m using it as the engine for the product I’m building right now (https://tygra.ai/).

Understood. Makes sense.

Give qwen3:30b and gpt-oss:20b a try. You don't need a fancy GPU, just a modern CPU. Those models are already pretty solid for analyzing your personal data.

That's dark, baby!

You can buy a good laptop and run open-source LLMs locally. 100% private. Disable its network. Stay offline.

Time to stop feeding the machine!

YOU CAN!

;)

Nice.

Of course, you need decent hardware to run LLMs locally, but you don’t need a super high-end computer to host qwen3:30b or gpt-oss:20b. You don’t even need a GPU for those models, as long as you’ve got a modern CPU. And they’re already pretty solid for writing and coding.

Obviously you need decent hardware to run LLMs locally, but you don’t need a super high-end computer just to host qwen3:30b or gpt-oss:20b. Those models are already pretty solid for writing and coding.

Gotcha.

Sam Altman actually said this on a podcast with Theo Von recently.

“So if you go talk to ChatGPT about your most sensitive stuff and then there’s like a lawsuit or whatever, we could be required to produce that,” Sam told Theo.

He even asked Theo about his own ChatGPT usage, and Theo admitted he doesn’t use it much because of privacy concerns.

Sam’s response:

“I think it makes sense... to really want the privacy clarity before you use [ChatGPT] a lot — like the legal clarity.”

:O

No, I'm not talking about serious medical issues. I mean uploading your blood test results and asking an AI, "Give me specific nutrition, lifestyle, and supplementation recommendations."

For work, as an employee, sure, it's easy to say the company approved ChatGPT or Gemini, so you can go ahead and upload, for example, usage data to get a retention analysis. But what if you're the employer?

We are talking about sensitive data, like bank statements, blood test results, and citizenship documents.

Oh, wow.

"All these sites may as well have all my information." I don't think so. That's scary!

But you do have a choice.

If you want to upload your bank statements and ask an AI, "How did my spending habits change in Q2 compared to Q1?" or upload your blood test results and ask, “Give me specific nutrition, lifestyle, and supplementation recommendations," you can run LLMs locally. No data will leave your computer.

Work: Because your company is using Microsoft 365, you can upload customer data to ChatGPT - for example, for revenue reporting. Am I understanding you correctly?

Personal: So, you download and run open-source LLMs on your computer?

Good point.

But I'm not sure that in 3-5 years people will just give their sensitive financial, medical, or legal data to cloud AI providers without a second thought.

Oh yeah, I've been there!

Interesting.

If you can run an open-source LLM locally on your own computer, completely offline, and use it for legal, finance, or medical topics, would you still say no to that?

Wait… why would anyone even upload a file containing passwords or secret keys to ChatGPT or Gemini?!

Anonymization is a good idea.

Just last week I wanted to do some financial analysis with ChatGPT (GPT-5), but I wasn't comfortable uploading the CSV with all my transaction data. Instead, I used Qwen3 running locally on my MacBook to anonymize the data first, and then uploaded the sanitized version to ChatGPT.

¯\_(ツ)_/¯

So, you download and run open-source LLMs on your computer?

I think the solution is local AI, not another cloud AI.

Nice.

What software do you use to run LLMs locally?

I tried Ollama but found requests a bit slow since it doesn't seem to fully utilize my resources. LM Studio has been better for me. It uses the upstream GGML implementation, which feels more optimized. My problem with LM Studio is that they've added so many options and features lately that it takes a lot of configuration.

What's your favorite open model for writing and coding these days?

Not a wrapper of ChatGPT. I mean no cloud AI at all for sensitive data. A different approach: local AI. A desktop app where you can download the open models you want and run them locally. 100% private, no data ever leaves your computer or network. The real question is whether companies would pay for team features on top of that.

I don't think we're comparing apples to apples here. Using a mobile browser certainly opened the door to more tracking, but at the end of the day it was mostly about ads and profiling. With AI, the scale is very different. These systems can learn far more about you from the questions you ask and the data you provide. That makes privacy and data protection a much bigger concern than ever before.

I don't think this article is relevant to what we're discussing here.

It says that The New York Times (NYT) is suing OpenAI for alleged copyright infringement, claiming that ChatGPT was trained on its articles.

What am I missing?

:))

Good point. This is how we started Tygra (https://tygra.ai/): as an online solution with a web app. But due to privacy concerns in industries like finance, legal, and healthcare, we pivoted to an offline solution with a desktop app. Today, Tygra is a privacy-first AI document processing tool that helps companies parse and validate documents automatically — 100% locally and privately.

Ollama’s new app is now available for macOS and Windows. Yes, you need to install it, but the process is simple and straightforward, so I don’t see a problem here.

That's the idea! We use a sophisticated combination of vision models and LLMs to accurately parse, extract, and validate data from complex documents. Our mission is to outperform competing document understanding models in terms of accuracy and speed for specific document types, such as bank statements and bills of lading, among others.

Our pre-trained models are tailored to specific document types, such as bank statements, invoices, bills of lading, etc.

When using pre-trained models, you'll find that the majority of fields are readily available during the document review process, and no initial training is required. You can start extracting data immediately.