searealist

How does an ASIC manage to have 60x the memory bandwidth needed to achieve that speedup?

The models themselves support up to 1M. You are just charged more for all context over 400k. The 272k limit is just a client limitation to ensure you can never go over 400k since the maximum output size of the model is 128k.

Have you ever said "I'm starving"? Do you think that undermines the experience of people actually starving in war or famine?

Please don't act as the hyperbole police. People exaggerate all the time (I'm starving, etc). It's normal, and you are being a jerk to call them out.

MTP can give you small batches, but it is still WAY smaller than the batches you can get with prefill, which is limited only by the number of input tokens you have (but has diminishing returns on performance).

But:

1) It still makes no sense to upload the weights to the GPU with MTP as you are still bottlenecked by the weight upload.

2) I'm not sure MTP helps much with MoE models.

There are two phases to LLMs:

1) prefill

2) decode

For prefill, you are compute bound, and it is trivial to batch multiple input tokens together. When using cpu offload, software like llama.cpp will batch weight uploads with tokens that need those weights and perform work on the GPU. It works very well. With a large batch size and pcie5 you can get prefill speeds close to having all weights on the GPU.

For decode, you are bandwidth bound, and it is difficult to batch multiple output tokens together. There is no benefit to sending your weights to the GPU because even if it internally has insane bandwidth, you are still bottlenecked by system RAM (and adding a pcie5 upload would bottleneck it further). This is the number people usually talk about when they say they are getting a certain tk/s.

Kramnik ((ab)using his status as a former world champion) accused several other top level players to the point that one of them committed suicide.

There is no evidence of suicide, and he had multiple drugs in his system when he died.

he probably did cheat in this particular game

This is a fringe opinion at this point in time, even though it was obvious he did not cheat on day 1.

Sindorov, who won the candidates, and will be playing for the world championship, has also had accounts banned on chess.com for (presumably) cheating as a teenager. Do you think it would be ok if a large part of the chess world conspired to keep him out of tournaments. Do you think it would be ok if the world made jokes that he cheated using a sex toy? Do you think it would be ok for the largest youtube chess channel (GothamChess) to publish a video titled "Chess cheating device" featuring a thumbnail showing his face next to a butt plug? And all this while he was 19?

What happened to Hans is FAR worse than anything Kramnik did. But maybe you think it's ok because you find him to be an asshole?

It would apply equally to GPU or RAM inference as those are also bandwidth constrained on decode, so people already try to optimize for it.

I think they will understand, but it will be a new perspective.

My experience of working in the industry is that many of my peers have had great animosity and little sympathy for blue collar workers who lost their livelihood to globalization.

I quit over a year ago when one of your moderators banned me for reclaiming an allied unit that was attacking ground on my factory (he was mad at me).

I sincerely don’t know which side is lying, and I can certainly imagine situations someone might say that.

Just to clarify, many third wave shops do serve medium and dark roasts. My point is that third wave coffee shops often call a roast dark when they are actually medium or light. Additionally, I think only light roasts really qualify as third wave.

Most of the coffees you listed look like actual dark roasts.

"Full City Roast" with "lemon zest" in the flavor notes raises some red flags for me. Indeed, from the reviews: "Yum! I don't like dark roasts, and this is not a dark roast which is why I liked it. Lots of chocolate. Very smooth, not bitterness, a hint of fruit as it cools. I really enjoyed drinking this."

"Imunika Black" might also be an exception: the "blackberry" tasting note makes me think it isn't very dark.

I see more with citrus or other fruity descriptions, and I doubt those are dark roasts either.

This is actually a perfect example. I agree that you can find dark roasts at some third-wave shops, but they are _not_ third-wave coffee.

Our premier dark roast is a nod to the old style of roasting which is still favored throughout the world

In this case, they even go out of their way to disqualify it as a third-wave roast.

Here the first sentence of Third-wave coffee's wikipedia page:

Third-wave coffee is a term primarily in the United States coffee industry emphasizing higher quality, single-origin farms and light roast to bring out distinctive flavors.

I also strongly suspect your definition of "decently dark" is still below a medium roast. If not, I would be interested if you could cite a "decently dark" roast you have tried from a third-wave coffee shop.

There are two axes here:

- Beans

- Roast level

The variation from roasting is far greater than the variation from the beans, and all third-wave beans are lightly roasted. This results in a brew that is dominated by acids.

I've come across many cafes in San Francisco that only have light roast espresso that tastes like lemon juice.

There would be almost no black people at Harvard if they did not do it.

You don’t seriously think that is a possibility.

From what I can tell, it all boils down to one of two things:

- Criminal activity

- Support of Hamas

https://en.wikipedia.org/wiki/Students_for_Fair_Admissions_v...

Compare the numbers for MIT who is somewhat complying vs the Ivy leagues who are mostly not complying.

I agree my original claim is likely not true. I should have said >95% of Asian's SAT scores are higher than the median Black SAT score.

It's not just Black students over Asians, it is the entire gradient I outlined, and they continue to use race in admissions, they just obfuscate the reasons now.

The thing is, acceptance at these elite universities is _highly_ correlated with academic performance. It’s just that different racial groups are admitted in different ranges. You won’t find a single Asian at Harvard at or below the median Black SAT score. The range to be accepted as an Asian is also higher than Whites, which is higher than Latinos, which is higher than Blacks. How can you possibly explain this if it’s not a quota system based on race?

Do you think there's a specific (whether public or hidden) criterion being used to deny Asian students based on "leadership abilities"? Or do you think they're simply being held to higher standards or subject to an informal quota?

only had traffic violations according to what I read in the Texas Tribune

I don't think that is true. Do you have any examples?

Charges are dismissed all the time when there is guilt. It is not the same thing as innocent. There is a close to 100% chance he admitted guilt.

A LED TV basically lasts forever. An OLED TV will last maybe 10 years max. Often less in practice.

At the time, Lidar cost about $80k for the main unit.

Do you think that is sustainable forever? If not, how long?