ghm2199

Words and sentences to an LLM are like witchcraft. There are certain words, sentences that make LLMs go a certain way and do vastly better. Sometimes its not at all apparent what set of words will work to do what you want it to do. An example I have been using to do design at a high level is to say to claude.

```

A question is salient to the degree that its answer changes what we do next. Operationally, saliency = the product of four things:

- Decision-leverage — would resolving it one way vs another force a different design or invalidate a stated decision? (No leverage → drop, however interesting.)

- Residual uncertainty given current evidence — is it still genuinely open after reading the docs and the code? (Already settled → drop, however deep.)

- Load-bearing-ness — how much rests on the premise.

- Cost of finding out late — architecture-deciding / expensive-to-unwind raises priority; cheap-to-fix-later lowers it.

```

There are a few things to note about this prompt

1. There is no reason from looking at it that it should work, it even has the word load-bearing which people loathe, but it remarkably produces a stable design with questions from claude (atleast from claude Opus 4.8 and even better from Fable5). Otherwise the design document claude likes to really write are implementation level(code or otherwise). I usually pair this with matt pocock's grilling skill to make claude behave.

2. From design -> implementation, its is generally about understanding when claude is trying to trick you into making something sound like a good/easy solution but has tons of untested assumptions. Here you have to read and patiently spot if a how you would get to the solution is not clear. A common error here are when claude makes a big deal based on what it read and interpreted too seriously without questioning the assumptions. There are several more.

But it also comes down to your experience as a SWE, much like a mathematician's. The frustrating thing about it is, it feels tha a skilled mathematician working with AI can make them productive in ways that are more reliable as compared to a SWE (e.g. lean is deterministic and can provide very strong feedback and LLMs are very good at using that feedback). Maybe a mathematician can chime in on that?

If 1000's of years from now aliens discovers these menus while excavating buried human cities they will get confused about what evolutionary dissonance was going on looking at the phenotypical structure of creatures in hand drawn paintings juxtaposed with AI drawn menus. "Did these humans who drew these live in different cut off worlds that these same animals evolved to look so different from each other?", "Did they engage in active genetic engineering instead(because they lived only for a few 1000 years in organized societies)?" they would hypothesize and get confused.

Future alien confusion is all that Gen-AI pictures add. There is no value to them that is remotely redeeming/useful to us today.

Should we just call it like this is: marketing PR. There is a reason why the newer open weights models like kimi's don't do this kind of stuff. Kimi is maybe 6 months old so like Opus 4.7 level now, it could do this I presume but it has not to my knowledge. Why? Because the incentives of open-ai and anthropic are very different from people releasing open weights models, the former gang seems to do this now on a regular basis.

There are a few things that perplex me even more:

1. If you are going to eventually publicly release models that are trained to behave according your spec or AI-Constitution to maintain coherent behavior**, why on earth would you want to tell anyone it can do this?

2. Do they have another GPT 5.6 trained to not obey a different constitution/spec to do this kind of hacking? Because that makes no sense since you would never release it.

3. And if this is a constitution obeying model, I am also curious what they did to it to get it to do this hack without serious pushback from the model's training. Whenever I have tried to get codex/claude to do a vulnerability scan of my own servers it always refuses constantly.

** I know spec based training has its limitations, but its all we have and atleast one knows what the model's persona is and what its value system is. But there is no reason you would make one model do that while letting another one be a crazy hacker. Its well known if you fine tune a model to change one part of its personal other often unrelated parts of it suffer from safety issues.

First of all congrats on the launch.

I worked at an edtech startup with adaptive learning platforms in previous decades. One thing that is always difficult to surmount was that at the under K12 level were:

1. B2b it was hard to land large contracts(like getting an entire public school district to jump from large publishers like HMH or Wiley) because it turned out that the teachers themselves did not care about adaptivity when they had a thousand other things to worry about. But teachers are not the ones that make the deals. It's the school district board and The reps at large publishing just said we are also adaptive and no one would care about how you differentiated. Teachers and students really aren't the lot you have to convince in a b2b deal do you :)?

2. If you want the b2c route then the customer acquisition cost was way too high because again like to get teachers and students to actually adopt this to help them was really a hard sell, especially when the public school system has teachers that are so overburdened with other stuff.

I see that you mentioned charter schools for your pilot, which are not exactly public schools but I dunno of they come with the same set of challenges, but I wonder are these above two challenges are Also something you have had to face while scaling your product?

Put all of these trends together, and I think that in 10-15 years, we are going to have consumer PCs (and phones!) running models.

I love the idea of SaaS offering these at lower rates today integrated into what ever you do and be 100% private. But I think the key challenge to mass adoption is productizing them in a way which makes sense for people to pay money for. As a commodity a local model is useless unless combined with some capabilities important to me. A PC is inherently useful because of so many applications offered on it on it. How local LLMs would be useful as a product that is useful for mass market is not yet proven.

Congrats on shipping this. I love handy on my Mac, my phone for STT in situations where it’s not possible/poor performance of the native Model for STT(e.g apple’s thing is not upto scruff, like mistranslating words corresponding to a domain).

Noob question: How do you think about funding from a foundation(i have no clue if you need it or not, I do hope you have a way to get paid one way or another because handy is amazing) for maintenance of this? if you did or were going to get paid by asking for maintaining such a project what might be the kind of organizations you would look for to get supported and how would you do it?

I think the only way political will can fund nasa to realize these 1960 design ideas is an infinite capacity arch rival that threatens/render irrelavent either the dollar's supremacy or american power (and just those two, because apparently these days there is no "threat"/need to defend a higher cause, like the neo-liberal rules based system or democratic or human right values). Also that arch-rival that is probably/likely not china(practically speaking)

I’ve had the good fortune of taking two courses at Columbia in the Social Sciences department(Andrew Gelmam and Ben Goodrich teach there). I think they probably are right up there if not the best at trying to teach students how to work practically with Statistics(specifically Bayesian statistics). Though they have always lamented that most schools do a poor job of teaching it such that kids can apply it.

Data science is always a very overloaded term ever since it took off way back in the 2010s. One of, if not the most, durable definition of this that likely can also be the most valuable because it’ll probably land your jobs(even today) is being able to make decisions from looking at data that have an team wide(good IC like job security) scope at the least and company wide scope(very rich).

Building that intuition is incredibly difficult. It can be learned if one likes to solve and think about problems that way. Like for example you can get quite far with knowing how to use linear regression(for example coefficients of linear regression can be determined using a deterministic algorithm using linear algebra yet knowing the assumptions of linear expected value and constant or variance is more useful as is the knowledge of what probability model to use to define the random variable(hmm are these Bernoulli events or poison)).

How to do sampling(like using reservoir sampling when you have an infinite sample count e.g in a long running crowd sourced survey to not over or under sample buckets for calibration).

Or just rule of thumbs like how # of samples needed for moving decimal point on significance varies roughly as inverse of sqrt of N and probably much more in case of interacting factors.

I would like a book on that :)

For local tasks you can only give agents delegated that execute your deterministic read or write on an allowed set of files(e.g pi does this) and execute rights only on containers with no network access. That should get you 95% unblocked for most tasks you want to do with an LLM pretty safely.

You can do a brainstorming with web on a remote container prototyping based on that brainstorm on another container with no network access.

The one thing that is less trustworthy is using local agents for service management, you definitely want to have them scoped to dev/testing. I would never trust an agent to execute any command in production or sensitive data at all

Someone commented on this page that their main market are long term b2b contracts. If that’s true then what you are saying isn’t a problem.

An so like if a business wanted to home in on one very specific use case that could be hyper optimized by SFT, had really good support for updating and adding new features, on-Prem etc. that’s the kind of market they are in?

Nice. I want to do the same too. What process/workflow did you use to move all the websites you had given your email addresses to, to move to your proton email? I am guessing it will take several years, but I would like to start the move of my gmail.

I would rank it quite close to The worst company to work for tech wise. I've worked for a bit as a contract worker in their Eikon/Elektron product division(the Bloomberg equivalent).

They have "everything": silos, fiefdoms. Systems team that operated as if the devops model never happened. A CTO they hired @~2015/2016 that had no experience in tech whatsoever. A board which instituted and instructed a cost cutting policy by,for e.g. , but not limited to outsourcing every job possible with little to no tradeoff on quality of engineering. And this was way before AI was a thing.

This left staff engineers with no bargaining power to hire engineers to write good systems.

If you are considering a job there, I would recommend doing some due diligence.

If you did that then you are better off never being targeted by emails/messages by the company at all. It is in the benefit of the company to know that. Right now its a tedious unsubscribe process that requires me to keep doing it all the time and company that does not know just blasts everyone who signed up. Its a ridiculous thing to do.

To get an idea of the thing. I would find something similar to https://docs.nvidia.com/learning/physical-ai/getting-started...

Caveat here is you may not be able to use their environments or you may or may not have their kind or robots to train your roomba. But at-least you could get an idea of how RL training is done for robots like yours.

This would require an inversion of dynamics based on quantification and collective realization of a couple of things:

0. Emails suffer from a "misclassification" of intent issue on a time*attention scale. Imagine time of the day/week/year on one axis and their attention on email inbox on the other. Emails have to arrive at the right (x,y) point for a user to act on. But they rarely do.

1. Well being of a user is proportional to their current state of mind to receive an message from X. Which is proportional to how likely they are to listen what you have to say.

Both of these suggest a negotiation of messages between two parties, much like when a bartender asks you if you want a refill and you can say yes/no.

Like it seems one needs to re-think email from first principles here. One idea is to use a the idea of "theory of mind"(ToM). e.g. The ToM between me and a sender would be for both to know: "I am not as excited as you about your product launch, so sending it is a 'spam' from my PoV".

We could use two negotiating agent, e.g. my agent that knows what I care about now/today/1-week ago and negotiates with an aspirant sender's agent before they send me any messages. e.g. I could set a policy based (my ToM) for my agent like "Between 1-1:15PM every day I want to read about all product announcements I subscribed to for XYZ product type". My agent would go talk to the aspirant's sender agent and gets messages right then.

An alternative policy could be "I have some free time now, create a summary/gist of all announcements on products I might be interested in.". The agents would negotiate with the sender to do the same.

Signups emails would be to replaced by an agent which "creates" a ToM with sender on hard-stop dates. I would tell my agent : "I am interested in this logging service to compare different ones, I will not be interested once ENG-123 is closed" and mine would not just tell the sender that they are not interested when the time comes (which is when ENG-123 is closed).

Longer term policies would just age out any message negotiations because I don't like/care about those products anymore.

For indie developers like myself, I often use chat GPT desktop and Claude desktop for arbitrary tasks, though my main workhorse is a customized coding harness with CC daemons on my nas. With the apps, b I missed having access to my Nas server where my dev environment is. So I wrote a file system MCP and hosted it with a reverse proxy on my Truenas with auth0. I wanted access to it from all platforms CharGPT mobile, desktop. Same for CC.

For chatgpt desktop and Claude desktop my experience with MCPs connected to my home NAS is pretty poor. It(as in the app) often times out fetching data(even though there is no latency for serving the request in the logs), often the existing connection gets invalidated between 2 chat turns and chat gpt just moves on answering without the file in hand.

I am not using it for writing code, its mostly read only access to Fs. Has anyone surmounted these problems for this access patterns and written about how to build mcps to be reliable?

One thing I have always wanted to do is cancel an AI Agent executing remotely that I kicked off as it streamed its part by part response(part could words, list of urls or whatever you want the FE to display). A good example is web-researcher agent that searches and fetches web pages remotely and sends it back to the local sub-agent to summarize the results. This is something claude-code in the terminal does not quite provide. In Instant would this be trivial to build?

Here is how I built it in a WUI: I sent SSE events from Server -> Client streaming web-search progress, but then the client could update a `x` box on "parent" widget using the `id` from a SSE event using a simple REST call. The `id` could belong to parent web-search or to certain URLs which are being fetched. And then whatever is yielding your SSE lines would check the db would cancel the send(assuming it had not sent all the words already).

For people like me — who are kind of familiar with how react/jetpack compose/flutter like frameworks work — I recall using react-widget/composables which seamlessly update when these register to receive updates to the underlying datamodel. The persistence boundary in these apps was the app/device where it was running. The datamodel was local. You still had to worry about making the data updates to servers and back to get to other devices/apps.

Instant crosses that persistence boundary, your app can propagate updates to any one who has subscribed to the abstract datastore — which is on a server somewhere, so you the engineer don't have to write that code. Right?

But how is this different/better than things like, i wanna say, vercel/nextjs or the like that host similar infra?

I read the TCP patch they submitted for BSD linux. Maybe I don't understand it well enough, but optimizing the use of a fuzzer to discover vulnerabilities — while releasing a model is a threat for sure — sounds something reducible/generalizable to maze solving abilities like in ARC. Except here the problem's boundaries are well defined.

Its quite hard to believe why it took this much inference power ($20K i believe) to find the TCP and H264 class of exploits. I feel like its just the training data/harness based traces for security that might be the innovation here, not the model.

Here is thought, this is a fixed 3d environment and you lack training data or at least an algorithm to train. Why not use RL to learn good trajectories? Like build a 3d environment of your home/room and generate images and trajectories in a game engine to generate image data to pretrain/train it, then for each run hand label only promising trajectories i.e. where the robot actually did better cleaning. That might make it a good RL exercise. You could also place some physical flags in the room that when the camera gets close enough it gets rewarded to automate these trajectory rewards.

I would begin in one room to practice this.

So my home router, all my iot devices attached to it from printers to projectors, not to mention custom stacks like Lutron. BLE based locks, car key fobs.

All of these technically could have zero day vulnerabilities and people/companies who made it don't have the resources to buy 20000$ of tokens to go debug them... Maybe they don't care but if they do, what if they can't afford such models or get access in time.

I would like to know how can someone like me defend against them?

If you want to get a feel of what brutalist architecture is like up close, go to the Barbican in london if you can.

Its quite surreal. Very much in-your-face concrete exposure. Yet, to walk and experience it with your eyes is a study of contrasts: a giant, comparitively modern, greenhouse, has a glass roof open to the sky and yet many floors have no light or windows at all. And in the outdoor spaces, like the fountain/canal running through the complex the concrete will sort of be in the background and lets you focus on everything else: the water, the swans and the people around.

Juxtapose that to low hanging exposed concrete roofs and walls in closed passages could make one feel constrained/claustrophobic/yearning for light.

I've been using handy since a month and its awesome. I mainly use it with coding agents or when I don't want to type into text boxes. How is this different?

Part of the reason handy is awesome is because it uses some of the same rust infra for integrating with the model, so that actually makes it possible to use the code as a library in android or iOS. I have an android app that runs on a local model on the phone too using this.

O(1) What! What might bring it down to say 10's of ms? Looks like its some kind of optimizable wall that its 500 for everything.

Like with 10ms then online replication/backup — analogus to litestream for sqlite — but for in memory processes becomes feasible, no?

For postgres there are pg containers, we use them in pytest fixtures for 1000's of unit-tests running concurrently. I imagine you could run them for integration test purposes too. What kind of testing would you run with these that can't be run with pg containers or not covered by conventional testing?

I'll say this is still quite useful win for browser control usecases and also for debugging their crashes.

You are people. You are also in the minority.

People hate AI. This will make them hate it even more. Yet somehow the market has convinced/forced us to use their products even though we might not want to.

A recent nationwide poll[1] shows AI has a poorer approval rating than ICE — ICE! — probably due to their overlords being "those" SV types. Everyday AI features are being shoved down our throats. I can't even choose to not install Gemini related apps on my Android when I select "which apps to install" when booting a new phone.

But people are a weird bunch. They largely don't buy products aligning with their values. No one is jumping up and down for Graphene phones even if they had amazing privacy first software. People buy 6mi/gal hummers and iPhones for fashion, brand, money, convenience/function. The pain threshold of all bad effects still is not high enough to quit their products in a meaningful way. Values and privacy are way down in their list. I wish people would not buy/install AI related features by big tech and be more discerning, but that is likely a pipe dream.

[1] https://pos.org/wp-content/uploads/2026/03/260072-NBC-March-...