tibbar

My entire office of nerdy engineers has been gathering 'round to watch the world cup for an entire month. Judging by this, the world cup does seem to be of some interest to hackers. ;)

Training and serving large models does require increasingly more compute, though. (The Chinese labs have clearly found some massive optimizations, but my point was that you'd think at some point even those optimizations wouldn't be enough to keep up with exponentially increasing model sizes.)

I wonder how the Chinese labs are training a 3 trillion parameter model on what has to be vastly smaller compute resources. If the U.S. compute advantage is persistent, it's hard to imagine that Chinese labs will be able to keep pace forever, as a matter of physics, but... so far they seem to be doing just fine.

Yeah, it's really tempting to try to fundamentally change the way you interface with the world but it's rarely very sustainable. I've found that trying to change my social environment, and also build skills to make specific tasks easier, are more effective options.

Writing detailed specs is an important tool to help the LLM translate the idea in your head to code. The spec is an intermediate artifact.

The non-determinism is one of the relevant features of this layer of abstraction! And one can learn to validate that the translation is being done properly. Some of the tools you have include writing extremely detailed specs, generating visualizations of the internals of the tool, or (perhaps) reading the code, though that becomes less feasible with volume.

Basically it turns out that code is full of incidental details and what you really want is to verify the important parts, while receiving a guarantee that the vast tail of incidentals is handled "reasonably."

I was quite worried about having to code when I interviewed recently. A two- or three- year layoff is a lot. Turns out that it didn't really make much difference! After a few weeks of warm-up exercises, coding was as natural as ever and turned out to be the easier part of technical assessments. I guess a couple decades of muscle memory is hard to lose.

Now then, back to using Fable. It is doing work that previously took me months in an evening.

Anthropic's goals here are not just harm reduction, but to stop competitors from making bio discoveries using Claude.

Unbelievable in which direction?

I've had years in which most people in my immediate surroundings were sick for weeks or months (likely exacerbated by mold, school, and travel). Also years in which I never really got sick at all.

Getting sick that often is pretty debilitating.

This seems kind of weirdly confrontational? Elastic was founded by the guy who created elasticsearch. Why shouldn't he make a living selling services around the software he created? This is a terrific success story!

This feels overly cynical. My long-time friend took a job at Meta (over equally compelling financial alternatives) because the manager pitched the team and growth prospects well. (Meta turned out to be quite disappointing on these fronts. I never heard money as an important factor for joining or for leaving.)

In general, the kind of people who get an offer from any particular big tech company probably can get similar money elsewhere, so it's unlikely to be as big a factor as you suggest.

Given how many studies have built-in sampling bias or other surprising assumptions, I still welcome people gut-checking it vs their experience. (Plus, the stories are interesting, right?)

Having worked for a business that made a serious go of running everything out of stored procedures, I have to say that lack of version control was a huge problem and effectively limited all development to a single person who held all the rules in their head.

lack of version control, clunky language mechanics, performance issues, etc.

This has been tried, but thousand-line stored procedures are truly a nightmare.

The description of the interview seems like it was explicitly non-technical, though.

I mean, we're pretty deep into Westworld/Blade Runner-style scifi at this point. It's actually a crazy, mind-bending question to try to grasp what is going on with chatclaudini at this point. Regardless of what labels we choose or properties we choose to affirm, we're far too deep into uncanny valley for it to be very helpful.

LLMs often seem to have trouble determining the severity of a bug/incident/problem in a vacuum. If you run an LLM over 1000 items in parallel and ask "is this bad," it will come up with reasons for it to be bad way more than it might if it were considering all 1000 at the same time.

Scrolling down, a bunch of these seem to just be "the startup shut down after getting customers", which doesn't seem particularly scandalous to me?

Fair point that there might be some functional changes you can suggest, but I continue to suspect that by the time this PR hit GitHub, all the most important decisions have already been finalized.

First of all, redis is amazing, and your 4 month development process speaks to the fact that you've already designed and verified correctness super thoroughly.

... just speaking as someone who sometimes has to review very long PRs sometimes, though, I feel like 25% is a roughly normal level of "signal to noise." 5,000 lines of core logic is a LOT, and the tests and dependencies do still need to be read.

EDIT: I feel like the problem, as a reviewer, is processing 4 months of intensive research/development and providing useful feedback. At that point, there's probably not much major input you can have into the core architecture or strategy, so you're probably not providing much more than a bugbot at that point.

Oh wow, I didn't realize that Redis is still mostly just authored by antirez! (My understanding is that he had left for some time and then returned to the project.) That is, honestly, pretty amazing. Well, redis is great and clearly it's worked out.

I'm curious, do people around you use AI? Because in my own workplace, people use lots of AI, and they ship lots of PRs, which correspond to actual features on the roadmap. I've been doing this a long time, and there is a whole lotta stuff shipping. I'm a manager and in the handful of hours I have I'm shipping the equivalent of what I would have as a full-time eng years ago.

Reviewing 22,000 lines of code, even from antirez, with this complex of a feature set and minimal PR description sounds like a nightmare. One starts to see why major open-source software like Postgres tends to be developed on a mailing list, with intermediate design decisions discussed by the community, separate patches for different related features, incremental review, and then a spaced release cadence.

I was expecting to see some verbose LLM output, but actually the code has a distinctly hand-crafted feel. Nice to see! I'm not sure if "production ready" is a safe claim 7 commits in to a project ;)

$8M sounds like a lot, but (a) the cost of making a material financial mistake c an easily dwarf this, and (b) the cost of the engineers maintaining the system was likely about this expensive anyway. And infra is expensive when you're Uber. It all seems rather overblown to me.

Doesn't context cacheing mostly eliminate this problem? (I suppose for enough context the 90% discount is eventually a lot anyway)

Yes. For example you'll typically have a "budget" of 1-10k writes/sec. And a single heavy join can essentially take you offline. Even relatively modest enterprises typically need to shift some query patterns to OLAP/nosql/redis/etc. before very long.

Sure, but this isn't really an AMA thread [despite the offer to "answer any questions"]. This is about Sid's journey with (extremely advanced) cancer. Airing grievances about Gitlab is just out of place here, you gotta read the room.

The point is that ideally the models keep improving until they can solve problems people care about. Which is already partly true, but there are lots of problems that are still out of reach.