philipodonnell

This particular example aside, I don’t think it being derivative and simplified is necessarily bad. Libraries that are popular today were written for humans and reinforced by LLMs via training. It’s unlikely they represent the ideal interaction surface for an agent.

There was a study recently that LLms prefer resumes written by LLMs rather than by humans. Stands to reason they would prefer apis written by LLMs.

This is probably the early days of such intentionally simplified agentic semantic primitives like “DAG Workflow” where the answer for why not Temporal is that LLMs prefer different things than humans.

Is the difficulty that in high entropy situations, you can’t really tell whether it’s because the model is uncertain, or because of the options are so semantically similar that it doesn’t matter which one you choose? Like pure synonyms.

Despite being a different kind of writing, there are some interesting parallels with the article in what you wrote here

I’ve been experimenting with LiveStoreJS which uses a custom SQLite WASM binary for event sync, so for simplicity I’ve also used it for regular application data in browser and found no issues (yet). It surprised me that using a full database engine in memory could perform well vs native JS objects at scale but perhaps at scale is when it starts to shine. Just be wary of size limits beyond 16-20mb.

Of all the interface modalities available, CLIs seem like the most natural for copilots to work with. Lots of examples in the training data, universal interface for help, maps well to the sequential nature of token generation, similar syntax for different OSs… I can see them replacing skills and MCP et al from the model’s perspective.

How do they prove this? It sounds like the plaintiffs basically claimed they were rejected a bunch of times and since the resume had recognizable indicators of protected classes they must have been discriminated against?

Don’t get me wrong, I do this work, and Workdays statement of “we don’t use protected classes” instead of “we test our models to prove they are unbiased when given recognizable indicators of protected classes” is pretty telling. Because it’s hard and if you solved it you would be proud. If you don’t control for it it WILL discriminate. See Amazon’s experiment a decade ago.

I’m just really curious how all this plays out in front of a judge.

You should just need the AGENTS.md right?

Anyone have a link to the actual report?

This is a great use case for using an algorithmic difficulty ramp where it can really dial in that curve to solve for getting people to play longer over multiple sessions.

I’ve built lots of pre-LLM data processing pipelines like this and the more I read people putting “agents” into this kind of context the less they resemble agents like the Anthropics of the world defines and the more they just resemble functions. I wonder if eventually there won’t be a distinction and it’ll just be a way to make processing and branching nodes in a pipeline less deterministic when you need more flexibility than pure code-rules can give you.

Isn’t this an arbitrage opportunity? Offer to pay a fraction of the cost per token but accept that your tokens will only be processed when the batch window isn’t big enough, then resell that for a markup to people who need non-time sensitive inference?

What kind of customers are using this API? I’ve had many similar thoughts but I get hung up on the idea that customers are “developers” from a marketing standpoint, because those developers are developing something and that something is probably a bigger driver of utility that a truly generically developer tool like Cursor.

If I’m understanding this correctly, if you set off one bomb the explosion travels at a certain speed, but if you put a bunch of bombs in a circle and set them off one at a time, the very last explosion will be going a lot faster than the first one?

The contributors to the charity get a write off too

There’s a trope where the best white hat is a former black hat because they can recognize all the tricks, I wonder if training an LLM to be evil and then fine tuning it to be good will produce more secure code than the opposite?

FWIW I’ve been doing a lot of work with text to SQL and in that space verbosity in naming tables and columns matters a lot because it adds additional context about what data is in the table and how it can be used. Think “subs.id” versus “subscriptions.stripe_customer_id”.

Very cool! What did you use to make the ER diagram?

Hi. Maybe anecdotal, Brother MDC-J480DW, had it about 3 years, always bought third party ink, two weeks ago I had to replace the color ones and it started saying there was no ink even though ink was clearly visible, it failed on two different colors from two different sources, and the three different black cartridges, then bought legit from Walmart and worked perfectly. I can’t speak for everyone’s experience, but mine was definitely changed recently to always say third party ink was empty, and I’ll junk it once this ink runs out.

I’ve been working on some tooling to create better views of analytics based on the idea that people don’t like to write joins so why not create logical views that do all possible joins. Appreciate any feedback. https://github.com/eloquentanalytics/pyeloquent

I think a lot of times this is not about skill sets but more that data engineers don’t build datasets with UX in mind. The examples in this piece are not what show up when a leader browses a real database with hundreds of tables stuffed with abbreviations and numeric/short codes. If you want your leaders to use your data, you have to design your data to be used by leaders, not teach them SQL and statistics.

I often wonder if you can create databases like this using the data in the training datasets of LLMs. Generate the list of spices by asking for categories, countries of origin, etc and then asking about a list of properties for each. You could use these kinds of Wikipedia lists as a validation mechanic.

Enhancing the comments on the existing data model seems to be the most common approach for sure. I'm implementing this as a data architecture at several clients and I've found creating a whole new logical structure designed for the LLM is really effective. Not being bound by the original data model lets you solve several problems related to the "n-hops" question, avoiding needing the comments, and the semantics of how data engineers define columns. Some more details here [1], but obviously you can implement this totally yourself by hand.

[1] (https://github.com/eloquentanalytics/pyeloquent/blob/main/RE...)

I’ve been doing a lot of work on semantic data architecture that better supports LLM analytics, did you use any framework or methodology to decide how exactly to present the data/metadata to the LLM context to allow it to make decisions?

This thread was wild.

Is OpenAI as a company more like a publisher or a model?

I often wonder if NL-SQL tasks would benefit from an intermediate query language that is more compatible with the next-logical token approach that is used to generate the code. Obviously there is less of this in the training set, but if it transpires in a testable way, you could generate training data yourself from known good sql queries? Are there any languages that have been designed specifically for this?

I’ve thought about this for a few years. Sq ft gardening would lend itself very well to a grid like this one. I wonder if anyone will find anything.

Honestly you should for sure get a job. You seem to be choosing startup ideas that people with jobs choose, use offshore labor from the start, rapid pivots into tools to pivot faster, udemy courses, these are all side projects. Nothing wrong with that, but do them at night and save up your cash for when you have a big idea that can’t be done that way.

I used to do this kind of work, not with McKinsey. When you do it a lot you realize how hard it is and why. Everything you want to measure, someone will claim it doesn't capture this and that, or will definitely be gamed and destroy the company. A lot like this article and discussion really. Many times picking metrics are the easiest part compared to getting the data, calculating the time series, setting targets, managing under/out performance. It's all kinda inevitable... so my advice is always to not focus too much on the metrics and focus on managing instead.

- Measure everything you your systems allow you to, maybe tweak some workflows, but don't add complexity or cost just for metrics.

- Separate metrics into stuff that needs to stay the same (mostly operational efficiency metrics) and stuff that needs to change (strategic metrics) and spend more time on the latter.

- Don't compensate or reward people based on them, just look at them a lot and if you see something that looks weird then talk to your managers about why, and if they can't answer or won't take action then that's all you need to know to reward them.

- If you want to use metrics to create competition just publish a leaderboard and let peer pressure do the rest. People will work hard to be at the top of a public list even if there is no explicit reward for it.

Just be a manager, manage the people, and use metrics that help you manage the people. You won't sell many consulting engagements that way tho.

Don't forget "intel on if/how your competitors have solved them"