boroboro4

The issue is it’s cpu compute which is underutilized in gpu clusters anyway, so practically it’s not really 1/1000.

It's very unclear what's special in Rubin to be optimized for inference? I can see disaggregated bit (with having separate prefill and decoding nodes), but what else?

What does it mean it's better at nvfp4 training? What's different between training and inference to make this true?

It’s crazy something which basically changes the way US government works on a such a deep level is not even on a main page of hackernews.

The core insight there is to separate value semantics (no identity) from reference (itself) semantics (nullability). While this particular change can bring very limited amount of improvements it’s still does some - probably smaller to no object header + more guaranteed optimizations for variables on stack.

It’s when they land next part (nullability) it will shine fully - particularly on the intersection of not null and value. Alternatively if they introduce tearable semantics it will also shine - it would be possible to still optimize array of value classes, even if they are nullable (for example by having correspondent nullability mask).

So they are taking right step in a right direction. They are just trying to land this incrementally.

There is obvious proxy to the amount of training data - revenue. And I think anthropic is way ahead of them.

DOJ puts an accusation with clarifying text in semi private document? They don’t do this, they do much worse things (and get, rightly, much worse response).

This document isn’t great, but comparing it to Trump administration actions isn’t great either. As well as focusing on it rather than on a substance of the article in question (which is, about Garry Tan accusations in a first place).

Thank you for your comment though it made me go back and reread the linked text more critically.

I somewhat agree that the whole document is sloppy, but I also think this conversation overemphasizes what is wrong with it. It is not as if the document simply accuses the journalist and leaves it there; it actually elaborates on what it means by “violated HIPAA” in that context (and please don’t make me start comparing it with current administration behavior).

There is also a broader question of how to properly handle what are, quite honestly, likely bad-faith actors — the journalist in this case. Should the office simply ignore the smearing campaign and the lies? Maybe. But this is already an issue even before accounting for the significant amount of money — pushed by a very small number of people — being spent against the attorney’s office.

But this particular one does?..

This is what great reporting looks like: well-written, transparent, and rigorous. It’s sad to see how hatred toward progressives can distort people’s judgment.

It's unclear why the most probable next token given the context "please pick random number" won't be distributed uniformly across all the possible numbers (in the end it's totally possible for LLM return 10 logits of around same value for numbers 0..9 for example).

My bad.

However billionaires don’t own tiny part of US wealth, more like 5%-10%. And top 1% (and grandparent was talking about rich people) own 1/3 of US wealth.

Total US wealth is ~170T so obviously it will be enough to cover federal and state government for a year (and more like 20 years).

Even considering obvious issue of wealth going down like crazy in such hypothetical scenario in its ends this would be enough. Because in the end it’s all part of same economy.

While I mostly agree with you, it worth noting modern llms are trained on 10-20-30T of tokens which is quite comparable to their size (especially given how compressible the data is)

This https://github.com/niklas-heer/speed-comparison/blob/master/... and this https://github.com/niklas-heer/speed-comparison/blob/master/... from parent comment

It's not an issue of warmup time, it's an issue of jit compilation.

On my server (AMD EPYC 7252): 1) base time of the java program from the repo is 3.23s (which is ~2 worse than the one in linked page, so I assume my cpu is about 2 slower, and corresponding best c++ result will be ~450ms 2) if you count from inside of java program you get 3.17s (so about 60ms of overhead) 3) but if you run it 10 times (inside of same java program) you cut this time to 1570ms

It's still much slower than c++ version, but it's between rust and go. And this is not me optimizing something, it's only measuring things correctly.

update: running vector version of java code from same repo brings runtime to 392ms which is literally fastest out of all solutions including c++.

update2: ran c++ version on same hardware, it takes 400ms, so I would say it's fair to say c++ and vectorized java are on par (and given "allows vectorization" comment in cpp code I assume that's the best one can get out of it).

This one doesn’t even have warmup for Java, which makes results complete non sense.

Those benchmarks should be just forbidden for their misleading nature.

He’s not a king to do whatever he promised as is, he’s bound by laws and constitution (which are passed by congress).

Also as you were corrected there is constant goalpost moving in terms of whom exactly should be deported and how.

If you’re really interested in public opinion people don’t support ICE and especially how do they do what they do.

You can find stats including pending charges: https://bsky.app/profile/reichlinmelnick.bsky.social/post/3m... the main uptick in recent arrests is mostly people without any criminal charges including pending.

You can see that a lot of charges aren’t that “criminal” too - it’s traffic violations or immigration itself.

There might be more than one reason for an ongoing crisis, and different takes on who’s responsible. However Maduro is responsible of huge number of refugees fleeing Venezuela, and we (and some other countries around) have some obligations to help asylum seekers.

Obviously the one which sets the law, also the one which has first article dedicated to it.

Language is just a form, what exactly is encoded inside of the model can be very different. And to encode logical reasoning inside of the weights with activation functions is more than possible.

Models solving IMO level problems imo proves it.

I also think you greatly overestimate human intelligence, the fact we got AGI is nothing but barely side effect of evolution.

This being said in this setup of 2-4 h100 you’ll be able to generate with batch size of somewhere around 128 ie its 128 humans and not one. And just like that difference in efficiency isn’t that high anymore.

Issue with your original statement is that it implies negative correlation between intellect and being good politician. All the issues you describe (both being self serving and power seeking) apply to anyone regardless to their intellect (and I still think they apply less to people of high intellect just because they see bigger picture, but I might be wrong).

We need to optimize for less self serving and more integrity but we should strive for smarter people up there too.

In my opinion even charisma implies higher than average intellect which is already something.

We should put more pressure on elected politicians around competence and integrity, sure, but it doesn’t mean random person is going to be better.

In the original comparison second category of people have much higher intellect than average.

Would you rather be treated (medically) by the first 2000 people? Do you think code will be written better by the first 2000? I get being unhappy about current political class, but this kind of claims is wild to me.

Mechanics is exactly the same - it's not Tesla revenues driving returns for investors, it's new investors putting their money into the stock at very high price.

From the comment below:

Groq raised $750 million at a valuation of about $6.9 billion three months ago. Investors in the round included Blackrock and Neuberger Berman, as well as Samsung, Cisco, Altimeter and 1789 Capital, where Donald Trump Jr. is a partner.

Makes it very hard not to think of this as a way to give money to the current administration. I know, this sounds conspiracy theory grade, but 20b is too much for groq.

To me intellect has two parts to it: "creativity" and "correctness". And from this perspective random sampler is infinitely "creative" - over (infinite) time it can come up with answer to any given problem. And from this perspective it does feel natural that base models are more "creative" (because that's what being measured in the paper), while RL models are more "correct" (that's a slope of the curve from the paper).

Probably if you use a lot of Arc<Mutex<Box<T>>> languages with proper runtime (like Go or Java) are gonna be more performant, in the end they are built with those abstractions in mind. So the question isn’t only how much the nature of the problem it is, but also how common the problem is, and is rust a correct way to solve this problem.