FridgeSeal

Let’s not forget to count all the people who use a MS office product at work, and are therefore opted into counting for copilot numbers, which almost _certainly_ counts towards ChatGPT numbers by default!

If exploitation is the game, ad-tech companies will find a way!

Talaas have done this with a llama 3 model. Runs at like, 16k/tokens a second oror something obscene. Very little power draw too.

Doesn’t need hbm or lots of memory, because the hardware can just forward the data straight to the next layer and you don’t need to round trip through memory.

They claim to be working on an approach to make the underlying hardware a bit more reusable between models.

This reads an awful lot like post-how justification of poor business practices.

It’s got a bit of”uhhmm actually, poor management and high turnover is good actually” vibes, which is then (over)extended to a kind of carte-Blanche justification of “why using kms and having no idea of what’s going on” is good-and-desireable.

Which is like, certainly a take, and I can think of at least one “technical skills hating” exec from a past life who’d read this and foam at the mouth to feel justified in their decisions to try and throw all engineering practices out with the proverbial bath water.

It sounds like you’ve got a super interesting stack going on there, evidently with a large performance/latency focus.

May (haha) I ask what this is in service of? I’m somewhat a fan of the thread-per-core model, so I’m curious as to what you’re doing with it.

Increasing threads beyond system's hardware cores/threads resulted only in marginal gains of a couple of milliseconds worth of differences on huge workload with large increase in memory

Careful, if you say that too loudly, the "get rid of async just spawn more threads!!!!!" people will come out of the woodwork to yell at you about how _all_ async is a lie and we should instead pretend none of it exists and just spawn more threads.

but if it falls prey to these pitfalls, it would make me pause before adopting it.

This issue isn't really a Tokio concern, it's pretty straightforward to write Rust code that has back pressure mechanisms. The "it's not reasonable" in my mind implies that if someone goes and _does that_ then there's not much a library can do to restrain the developer.

This reads an awful lot like the prompter had a semi-confusing time with some async, and had their favourite model write an upset blog post about it.

I don't think these systems are perfect, nor are they fit for every use-case, but some of the complaints ring a bit hollow.

fetching a database record over the network, then immediately crunching the data. But what happens when that data crunching involves parsing a 10MB JSON payload

Mixing IO-sensitive code and blocking code causes issues, who knew? I'm not quite sure how these libraries are supposed to magically _save_ you from this?

When these latency spikes occur, the answer is always the same: separate your runtimes.

Well yeah. "Why doesn't my daily-driver cut sick lap times around the Nurburgring?" If you want to run blocking, non-interleaved code, don't do it in an executor expecting small, interleaved, non-blocking tasks. The docs for Tokio even mention this, and provide a number of worked examples of integrating/bridging sync and async code.

If a developer must manually partition I/O and compute, strictly police the boundaries to prevent deadlocks, and ferry data between two different runtimes with two different mental models, the async abstraction has failed.

Not necessarily. If I'm chasing a performance target, and the tool gets me 80-90% of the way there, to the point where my next task is optimising layout and caching, I call that a win. That's performance ground we'd have to address at some point if we want to go faster, so getting there easily means: - those people who don't need to go faster because their perf requirements are met are happy - those people who need to go further get to skip the intervening work, and go straight to these optimisations.

Edit: also wanted to make a point about the "memory blowup" the author mentions. It's entirely possible to do the _same_ load-shedding the author wants in Rust/Tokio/Monoio/etc, this is again, just a matter of building your application in a way that uses, and communicates back-pressure. I gather that they appear to want something which is a bit more opinionated and "batteries included" as far as functionality goes, which is totally fine-and-cool, but calling Tokio + Rayon _bad_ because they explicitly don't do this is a bit of a miss.

That is indeed, the joke.

don’t understand why anyone thinks management of any kind was involved in this at all

Because it’s marketing gold? Any marketer worth their salt would be super keen to get their claws into a project like this.

Anthropic and friends are in then business of selling LLM’s. This is a demonstration of LLM capability, they’d be foolish to not even try to cash in on it.

Plenty of companies, very much _want_ this.

I agree, that work in these places is likely short lived, if for no other reason than working in them is awful and demoralising. My point is, that these places exist, and have such a hard-on for “oh my god, AI!!!1!1!” that putting in extra effort there, is a waste of your own energy.

I just don't care anymore because it's not my responsibility or my code at all at that point

Yep, 100%.

Business has made it clear they don’t care, so there’s no point in burning one’s energy. Throw the whole thing on auto, check out, and go do something else during the day.

Amusing that you think businesses still have QA teams.

If you encounter something that’s not an int, that could be a signal you have a bug in your writers

Which, is something you could have caught before it got written at all if you had your db enforcing your types.

Because how it’s made and functions is useful information for dealing with it when they’re trying to attack you?

I give coastal piracy about 3-weeks to figure out how to commandeer unmanned cargo ships. I bet they’d be ecstatic.

Not if there’s good money to be made exploiting people for money instead!

There’s plenty of evidence that these models worsen these issues, and we know from prior experience that companies like FB will happily exploit vulnerable users for money.

The “nuclear bomb vs coughing baby” meme comes to mind.

Well this is a new brand of whataboutism I haven’t seen before!

Did something bad? Better ablate yourself of the responsibility of holding people to account for making it _worse_! Acting this way just makes it seem more like you regret the blood on your hands because it has dirtied your shirt, and not because you’ve done something actually bad, otherwise you’d have at least some degree of guilt and reticence to see things get worse.

This is a cute argument to make, which patently doesn’t work in the face of decades of evidence on the pernicious effects of social media _alone_, let alone the “turbo infantilising sycophancy machine”.

Pretending that everything’s totally fine and ok because “it’s adults” really just disregards reality.

Somewhat of a self-fulfilling prophecy no?

- nobody else with the skills - talk to machine - nobody develops the skills to have a conversation - go to step 1. - bonus step: your own skills atrophy too.

“Who cares about dirty syringes being left in the street, it’s really important for my work that I have access to the heroin I need”.

Things in society don’t exist in a vacuum, and “who cares about them, so long as I get my needs” is quite literally a part of the problem being discussed.

This is repeatedly the argument rolled out by people desperate to get us to accept it at all costs.

Ai doesn’t technically stop me writing code. But it’s sucked so much joy and interest out of it, that I can’t scrape together the motivation to work on side projects anymore. It has a “chilling effect” the effects of which are notoriously indirect to observe.

Can you just take half the module, ctrl-x, ctrl-v into a new file, and get faster compilation times without having to do any massive refactors?…In Rust, almost always no, because of circular dependencies

This feels like a strange, overly-specific complaint. It reads a bit like “When I write entangled code, it’s hard to untangle”. Like, yeah, the only thing that’ll save you from that is…not writing entangled code? I’m not of the opinion that the argument of “yeah but C lets me do whacky stuff” is a particularly strong line.

FWIW, letting a module grow, and then splitting modules up by cut-and-pasting stuff out along natural domain lines generally _is_ how I write Rust. Largely due to how easy it makes it to construct modules and submodules.

What kind of silly take is this?

“You deserve to starve instead!!” - is this really the position you want to argue?

Anybody here think that Palantir is not a security risk for Spain?

It boggles the mind a bit, but I’ve seen a few comments on here with people defending them to the tune of “what’s the big deal, they just help governments with their data! They're innocent” which is uh, either aggressively naive, or just paid PR behaviour.

Nah name-and-shame them!

Forza Horizon 6 was a bit of a shit-fight to get it working. GPU crashes, audio just failing, and 20-questions with what combination of runtime and configs would get it to play ball, and it would break after some patches, but it really stabilised now and most issues I’ve had have disappeared.

I wish I could work somewhere where I’m _marginally_ less subject to the whims of the Boardroom class.

I’m sure they’re having a great time, and getting filthy rich doing it, but I don’t enjoy having my livelihood attached to the consequences of their repeatedly-stupid-behaviour.

Every day, the rate of “could post a picture of 40k tech priests and have it taken unironically” goes up, and it’s starting to get concerning.