robbie-c

While I'm here - I recently optimized our PR review skill to make better use of the KV cache.

Previously, it loaded up the diff, persona and prompt, and wrote those into the first message for each of 6 sub-agent reviewers. The prompt was templated with the persona name, so was slightly different for each reviewer.

My optimized version had a common first message with diff + prompt, and a script to run to atomically claim a persona. It also runs the first reviewer before the rest to warm the KV cache, and the agent doesn't launch the rest of the subagents until the first persona has been claimed (which means that the LLM is running, and therefore the KV cache is warm). The agent has a script it runs which does the waiting for it.

Agents 2-6 only run once the KV cache is warm, and because the intro message including the diff is shared, it's there in the cache. Only the persona file is different.

In my testing this brought down the cost of a review by ~half, though of course this depends on how big the diff is, how many agent are launched, and what model is used.

I'm begging people to write articles themselves rather than letting Claude do it for them.

I want an expert opinion, if I just wanted to ask an LLM I have my own.

How can this article not mention the KV cache even once?

There are people who do, - And there are those who criticize.

There’s no need for that. I’ve done plenty.

Is it on topic to complain about the various claude-isms in this article? I don't know any actual humans that write titles like "Two floors the rate card hides".

I find my brain disengages once I suspect something of being written by an LLM. If the author didn't put much effort into writing it, should I expect them to have put much effort into fact-checking it?

Edit: this specific title has been deleted from the article. That was not my point! Please put in more effort into writing things that you want others to read! Rather than putting in low effort but being better at hiding it.

They sort of are, in that they want subscription users to have clients that behave well with the KV cache etc.

If you don't use a subscription, and pay per token instead, you can easily move to another harness.

A family member stayed in a hotel in Dubai recently and on returning said how incredible the staff were and willing they were to help them, and my response was "no shit"

Nothing has changed, just OP found the mirror of the main repo with the ee/ folder removed. That repo has been up for most of the lifetime of the company.

PostHog has always been open source. I'm not sure why this has been shared on HN at all.

That's valid criticism, I kinda hand-waved the "months" part. I read everything I could about parsers while building this (I have a CS background but hadn't thought about parsers in a long time) and came across this blog post https://lakesail.com/blog/sql-parser-in-one-week/ which talked about building a toy parser in a week, so I scaled that up to months for a production one.

The previous parser is mostly a declarative grammar file, which is extremely readable. It codegens a C++ parser, which is hard to read. It depends which of those you count as the previous parser's source code!

In the future, we'd make changes by modifying the ANTLR parser first, then using the same approach as in the blog post to get the new parser to parity. We have no plans to get rid of the C++ parser as an oracle!

Yeah, one of the interesting parts to me while working on this is that the breakpoint for when it's worth writing your own parser vs accepting ANTLR's slowness has shifted massively. Previously it would have been someone's full-time job to maintain. Now with this approach you can get the best of both worlds.

:D

I was hoping to avoid the criticism that I was picking the most favourable number and being misleading!

In what way? This was a geometric mean of the improvements from a small test corpus. In production, where it only parses longer SQL that didn't hit the parser cache, the mean parse time went down by 454x, across millions of parses.

Ha I did consider that! But 70x is plenty fast enough (we still have to query an actual database!) and the parser runs in a shared process on untrusted input, so it wasn't worth the security risk

It took about 2 days to get a proof of concept, and about a week to get something I could ship to production.

I skipped a few features for the PoC (like XML tag support, token positions), so most of the delta was adding those back in!

Our SQL is very similar to ClickHouse SQL, in that we used ClickHouse SQL as a starting point as that's what our underlying DB is. We needed to have our own parser so that we could add additional language features on top.

We pay github quite a bit of money and it's down for us too

I did that first, which didn't work for a couple of reasons. Many "big and tall" stores are the intersection only, and many of the stores didn't quite have the right vibe. I do have some of my wardrobe from places like 2tall.com but I was looking for something very silly for an in-joke for a friends vacation.

Undecided. One of the sites didn't have an LT but the LLM flagged that chest dimensions on their large were narrower than others, so could be worth trying.

Huh, I shopped for clothes using AI today.

Not super relevant to the Googlebook ad, but in case the perspective is interesting to you: I'm quite tall (194cm) but not very wide, so I usually struggle with buying clothes online. I used AI to scrape a bunch of clothing stores to see whether they sold a men's shirt with an LT or slim fit size, in stock, and matching a particular vibe.

Wait - are you missing all the context on this? Anthropic pushed back against this hard, there was a whole back and forth. I'm on mobile and can't look it up for you atm but if you google about this scenario, Anthropic definitely come out of this looking a lot better than OpenAI and xAI

Quite a lot worse. Both OpenAI and xAI were among the largest donors of Trump's campaign

Musk was the largest individual political donor of the 2024 election [1] and Greg Brockman was the largest donor to Trump's "MAGA Inc" super PAC [2]

[1] https://www.washingtonpost.com/technology/2024/12/06/elon-mu...

[2] https://www.theverge.com/ai-artificial-intelligence/867947/o...

There's proof of financial dependence, here's a recent report https://assets.mozilla.net/annualreport/2021/mozilla-fdn-202...

In 2021 they got $500M "royalties" (this is their payment from Google) with only $75k revenue from all other sources, including $7.5k donations.

I think it's just an AI-generated simplification, sucks that it made it to the front page. The subject matter is interesting, I would have loved to have read something written by an expert!

I've wondered about this too. I live in the UK and have been idly daydreaming about my next startup, and it seems like Brexit, and therefore having a small market / uncooperative EU, is such a headwind for some of the things I'd like to do. Seems like many UK startups just pretend to be based in SF.

I think rejoin is going to be politically unpopular for a while, as there's no way we could rejoin the EU on anything like as good terms as we left on.

In contested elections both sides usually have a large amount of spending.

I think you're thinking about this in the wrong way.

What you're saying is that people who don't have a lot of money to spend usually don't make it to the election.

Disclaimer: IANAL

If the same IP address is hashed using the same method, the result will always be the same, meaning it can be matched.

The way people get around this is by using an ephemeral salt, that is deleted e.g. daily. After enough time has passed, it'd be impossible to reverse the hash as the salt would be lost.

Probably not, see https://www.anthropic.com/research/reasoning-models-dont-say...

It’s irrelevant in terms of the transfer of the title, but it might affect who gets to challenge the current champion. E.g. the opportunity might go to the competitor who’s won the most matches since they last fought the champion. Or it might go to who they think would sell the most tickets to the title match, and winning your previous matches would generate that hype.

That’s one way to measure it, sure. Another way is that archers can shoot 2 tiles away.

Part of the abstraction of the game is that the map for cities and the map for units are at different scales (despite being the same map), and scales change depending on what year you’re in