scosman

join me in building the ideal training set for pelicans riding bicycles: https://github.com/scosman/pelicans_riding_bicycles

IMO "end-to-end encryption" simply isn't possible

In a technical sense it's absolutely possible. Owning the servers != transferring keys to the servers. Most E2E apps run both client and servers, it's about if they ever had key access.

Go mobile definitely supports building libraries. It will package up an xcframework for iOS and something else for Android (I forget details). It doesn't need to be the whole app. You get nice(ish) language bindings.

That said, a pure c framework is probably a cleaner way to go.

small issue: xcodebuild will just randomly fail sometimes. And the solution is to... open and close Xcode. I wish this wasn't true, but it is.

ChatGPT has had this for over a year too. This is a better model.

What's the best "run your coding agent in apple containers" setup folks have?

heh, who knows. I often give agents the motivation, which I generally helpful. Have not bothered to measure this case.

The top of my ~/.claude/CLAUDE.md:

Don't turn a one-off or area-specific comment into a durable memory without my explicit confirmation. You have a history of over-indexing on one-offs, and those memories end up getting cited to override well-tuned skills.

Kimi K2.5 >> Haiku. Gemma 4 32b might fit the bill.

Look at Qwen for that level of intelligence.

short answer: they mostly aren't

A few people are running highly quantized models with limited context windows. It's still impressive, but not the benchmark level intelligence. Very few people could afford a rig for reasonable local performance at a reasonable quant, at full context size.

The antirez example is 2.6bit quant, 32k context, and few tokens per second... on a ~$7000 MacBook M5 (new RAM pricing).

I dunno. I know a lot of software engineering experts. AI isn't always right, but neither are the people, and it's getting better and better.

Software is one domain where it excels because of structured training data and simulation environments, so I'm well aware it's better here than other areas.

Still there's somewhere balanced between saying every time it's "insufficient or incomplete or outright misleading" and "just trust AI". AI's a useful source of information/reasoning/research, but know you need to validate it's answers for important decisions.

This was copying many screens, composing layouts and many strings. not a "general look and feel" or one off utility string. Others found parts of the code copied (exact variable names). It would take a court to decide, but I think it's pretty clear cut copyright violation. They gave their agent had access to the EE code, and seemingly asked it to produce a copy. It's not a 1:1 copy, but there was a lot of copying. No clean-room attempt was even made.

If I take the first chapter of Harry Potter and write a new ending, it's still copyright violation. They took many parts.

They copied strings, and visual design. Both of which are work, both of which are covered by copyright.

Short segments of popular works sure. Many UI pages with identical layouts and copy, essentially zero chance. The agent had access to the original code at inference time.

"ghostlygourd.com" is a S+ tier domain. Would click

It's not economic to run them locally. It's amazing for privacy, and fun hobby. But you're either looking at super slow CPU builds with $10k in RAM, $90k worth of GPUs, or a really quantized model that doesn't compare in quality.

I might build one for fun, but it's not going to change the economics alone. Still exciting it's possible.

anyone manage to get a point?

Yeah totally agree! My pet peeve feature list is also better for society than donating $500M to charity!

Top right: link to the Lore Repo on Github

My 8yo son plays with this any chance he gets. We've started building watches together as a result (simple "drop in movement mods", but aspirations of building our own movement). The author really made something special.

I think the point is: if every app chooses the best model for their use case, my phone is hosed (disk, memory swap, memory). A good-enough default might be better for the user than the each app having the best possible.

ahh yes. I'm using Whisper v3 turbo via WhisperKit as well. Will play with parakeet

I think that's what they are trying to avoid. If you need on-device intelligence, their pitch was "The model the device already has is best", and if you need something more specific an adapter (aka, a fine-tune/lora) is best.

They were wrong when their on-device model was way behind. They still might be right in the long term.

While multiple app I use might need Gemma 4 E4B, I use dozens of apps and app devs can choose from hundreds of models. A shared cache might reduce size a little when there's overlap, but the core problem still exists. If each app chooses a model disk and memory-swapping explode.

Its probably be better for device manufacturers to bake in a default. I'm not proposing they limit you from using others, but one shared default might be best developer/user experience for 99% of apps.

- Being warm in memory is the single biggest perf speedup you can get, and a default is much more likely to be warm.

- "Best model" is usually "best model for this device" given both RAM and compute. A developer can't test every device but Apple can/will.

- Each model needs to be optimized for the hardware (what's running on ANE, what's running on Metal, what's running on CPU). The default gets optimized.

- If you need custom model, a Lora is probably best (30MB, benefits from all of the above)

You could say the default should be swappable, but that's more a linux ideal than an Apple one so I doubt we ever see that. Plus there are real downsides: intentional or not, prompts end up optimized to the model they are developed for, so swapping the default system model would degrade every app.

I had the same experience so started building my own. All problems are solvable, just working on the polish.

- crash recovery: part one is use ADTS aac (even if process crashes, audio is saved up until it does). Part two is isolating the transcription/summaries in separate XPC services.

- disk space: AAC 64kbps mono soles it. Could use Opus for further reduction but both are small.

- speaker bleed: macOS voice isolation processing solves this. It’s a nightmare to get setup, but works great once done.

- library: using argmax SDK - by a bunch of ex-Apple on device AI folks.

It it wasn’t for CoreAudio, I’d say it was easy to make. Argmax, Whisper, and llama.cpp - wrapped in the right architecture, mostly just work.

I’m having fun nerding out on the details like custom vocabulary (get the names of the people in here meeting right), inferring speaker names from transcript, calendar integration, nice UI, etc.

Those transcription times are fast fast. What model/library do you use?

They actively use the name https://www.war.gov

This is likely correct, sorry for the bad info. Was working from memory.

Free server-size model access for apps with <2M downloads, getting the same privacy guarantees. Hopefully they scale this up to all apps in time (I assume hardware/cost constrained, but larger devs would pay).

https://developer.apple.com/private-cloud-compute/

just a privacy-polished frontend for Gemini

I'd use this.

I'd rather have strong privacy guarantees, but this is still good.