augment_me

Just one more harness bro, one more harness I swear one more harness and we will have solved AGI bro

Peer review WAS vital for a long time. Maybe the world looks different now, maybe LLMs can find value in things better than humans. When you make an assumption it's good to think about why you do so, in this case it seems to be for historical reasons.

You are then our of the normal probability distribution and out of luck, it's not profitable to cater for you for the company.

My comment is aimed to highlight that the "GPU Bubble" is frames as a general solution when it's not, its a specific bottleneck based on your model size. Your dont mention your model size anywhere, the reader has to infer it from the runtimes, and if they dont know the average forward pass of a model, well too bad, they will leave without understanding the actual trade-off.

The benchmarks you point to in the section titled "A cost model for the bubble" dont include any CPU overheads or the T_block-T_pipe you mention, they just give the improvement %.

In general, you answers here in the thread read as defensive and unhumble. They leave a sour taste of your company, you should consider how you engage with your audience.

As someone who works in the field, the blog is nice but it has a lot of CODEX fingerprints on it, and it's also very specific to the size of the model in question in a way that is not explicit from the blog until the very last section.

In general, for some reason CODEX loves CUDA-streams, it's the first optimization it goes for every time when writing GPU kernels. However in many cases this is not a bottleneck, it happens to be so here because the model in the blog is small (2.4ms FW-pass is tiny, and 9B params sit on a single GPU). Large models are closer to 30-40ms. The CPU-GPU sync is 1-2ms, when working on larger MoE models the scheduling of tokens in this way is much less important than for example scheduling of computation/communication or kernel optimization.

I wish the blog would state this at the start with the premise of what has been done, or show that this is indeed the bottleneck with some benchmarking. Otherwise is kind of overselling things imo.

Whether brains do gradient descent is irrelevant to a CFO deciding whether to staff one radiologist or three. The market doesn't care if the model "identifies" versus "returns a statistical value", it cares what the error rate costs versus what the salary costs, and it prices liability into contracts either way, same as it already does with autopilot and compliance software.

Also disagreement among human radiologists has been documented for decades, so the clean expert baseline you're defending doesn't actually exist outside this argument.

When the identifiers pass human-detection-rate percentages, it will most likely be cheaper to hire a fall-guy for the liability with a much smaller salary, I think this will be a big market in fact.

You could also view it from the perspective of that if every other major superpower has their mass surveillance and you don't, it becomes an assymetrical informational situation where foreign governments can influence your citizens, but you cannot influence the foreign citizens since they are surveilled and their informational diet is restricted.

In some sense Chat Control is a geopolitical necessity for the EU, there is no choice to not do it.

If you have a confounding variable or a dependency that influences the experiment to a degree that invalidates the premise of the experiment, you need to put more weight on this in the conclusion.

For me this reads a bit like if I added an AI software that scans for shoplifters, and then placed a security guard at the exit of the store that watches the people shopping at the same time, and then said that the AI software is responsible for the reduction of the shoplifting without accounting for the influence of the guard.

If you have place the model in the embedding space of 99% negative samples, it's doing the same thing, the initial premise of the experiment is not valid.

LLMs produce about 95% of the code at my company and review about 70% of it for 3 years now. Our team has downsized from 40 to 8 people in this time. My creative labor is spent writing harnesses and wrappers. When there is enough of a data distribution on this, the LLMs will be able to do that as well.

I have saved up a buffer in funds and bonds because it's going to be over at some point when the company moves from explore to exploit.

Both were noted, but then the conclusion drawn from these things is that the author is considerably more optimistic about the agents. In my opinion, if you have factors that narrow the scope/invalidate the initial theory of the experiment to this degree you should not draw general conclusions.

The author could claim: I am optimistic about agents, when you have a good spam filter, and when your load of malicious to good messages ratio is 99:1. This is quite different from a common scenario where this would be used.

Who cares if the consumer buys it and uses it? Information is worth nothing anymore, attention is, so if they manage to capture a larger audience somehow, they win.

1) Googles spam filter removed a lot of the attempts as you say yourself. 2) Model was tested under unrealistic conditions where 99% of the inputs are malicious, so the model is expecting to get hacked and is already in the cautious part of the embedding space.

I know it's hard to account for everything, but in my opinion this mostly showed that the first 3 attempts were unsuccessful.

Which is interesting given the Mojo blogs where they shit on the other pythonic eDSLs like Triton saying that it's a dead end

Bitter truth :(

Most people use social media such as discord or whatsapp in order to make social activities and communities simple with the majority of their friends. A majority of people do not give a shit about integrity. The only group I have ever managed to convince and actually use Signal for messaging out of all my groups of friends and peers is the Computer Science Dept PhD students.

For most people, it's not realistic to give up their social bonds, they are too far in. If you are hoping for some revolution or change in this aspect is way too late. You can have small fringe groups engaging in this, but at the end of the day you are overestimating how many people actually give a shit.

The correct solution is just to not attend it if you know you aren't requested to participate and are just here to grow the numbers and make your company waste money.

If this argument actually worked in practice, the world would be a better place

I feel like the whole blog and the point can be reversed. If your bottlenecks are meetings and emails, and you make an agent take notes and summarize things for you, you gained focus to work on what you find meaningful.

He explains that this happens because knowledge work often relies on “pseudo productivity,” where visible busyness is treated as a proxy for real value. Digital tools reinforce this by making people look active: sending more messages, producing more drafts, attending more meetings, and generating more work artifacts. To avoid the trap, he recommends measuring real outcomes, identifying the true bottlenecks in one’s work, and separating deep work from shallow work so that digital tools support meaningful progress instead of consuming attention.

---

Like, you are just as well make the argument that if you replace the pseudo-work, you end up with 8 hours of deep work for things that bring you value.

TLDR:

Authors realize that global row-wise dependent functions like RMSNorm/LayerNorm have baked-in scales that are commutative in certain setups, so they can be moved out after a subsequent projection and be partially aggregated on tiles of rows.

So ((W1 @ gamma * globally_computed_scale) * W2 can be written as (W1 @ gamma * W2) * globally_computed_scale as long as we have row-only interactions for the scale.

This was usually not done before because left-to-right graph compilers like torch.compile can't assume that a global row-wise reduction between GEMMs can be commutative.

And the point of the comment you are answering is that the market you are talking about has taken on a different form.

The difference is that normally, people would perhaps pay a company to buy a PS license, or pay a professional to edit their images, however now this market segment will just use VLMs to edit the image to what they need.

I shop for things with AI as well. For example for haircare, or skincare, there is no way to figure out what ingredients are fine in various products. I pulled down 600 shampoos, prices and their ingredients, and made the AI choose which one based on my hair type and what I want.

I have another pipeline that pulls down all the groceries from stores every week in a 3km-radius and then builds cheap, healthy recipes from them, then orders the things I need by how the stores are laid out.

In general I spend about 65% of what I used to, so I think that the incentives for consumers are there.

I think that if this becomes an actual problem, there will be such a massive incentive to add AI to the scale/compression/risk avoidance side that there will be automated tools specialized in that kind of work.

I feel like this is shooting from the hip from a single point of view from some semi-large corpo.

That's completely your subjective opinion that ignores reality. If people feel like they are participating in something, or they feel like their identity is based on something they consume passively, it's as valid as the physical thing.

If people did not feel good from passive consumption, no-one would be listening, following or looking at things, people would just make and create all the time, which is obviously not true.

If what you say is true, there would be no value from AI-generating blogs in question, or AI-generated movies/youtube films. Yet both have millions of downloads, views and listens, as the article mentions.

The same way that the AI generated podcast about knitting, or engaging in consumption is enjoyable for many people and a form of stress relief, which was the point that the comment above was criticizing.

So the conclusion is that the utility of the activity is subjective, and if most people spend their time listening to AI factually incorrect podcasts about knitting and enjoying it, it's no different than knitting yourself and enjoying it. The blog was poor in this disambiguation, and pushed a more Aristotole-like ontological view of what is meaningful, which is more common view in engineering/hard-science dominated fields.

Look, I get your comparison and while extreme, it's funny. I just have very little faith in that the average person cares this deeply about the physically grounded reality. It's kind of a luxury of the well-off to be able to sit and think about what content to engage with when you just want to relax after a 8 hour shift followed by picking up kids, getting groceries, etc. If someone sees an AI-video that makes them happy or laugh, they send it to their friend who also laughs about it, that's their reality.

We happen to have time to argue about the philosophy about direction of the ontology of information at the downvoted bottom of a HN thread today, most people dont.

Your meaning and your truth, not necessarily other peoples who find their meaning and truth in other things.

Go to China, or Congo and you will find that the public might hold a different version of some truths than you do.

We had religions dominating the world order for thousands of years, which projected their versions of the truth onto their societies.

If we would extrapolate that to today and to your opinion, it would be that everyone in the middle ages actually had it all figured out, they knew that the religious texts about splitting oceans or the moon were fake, and were all just playing along with it for the social structure.

Maybe it just happens that the LLM-generated stuff is the next thing in this iteration.

It's interesting how every time this argument is made, its about subjective experiences of 'craving'. If this was the objective reality, we would have a majority of Gen Z engaged in movements, social groups and other concepts that would help them fulfill their 'cravings'.

However, it seems to not be the case, it seems like they prefer to spend their free time to doomscroll, or sit at home, and engage more in parasocial relationships that perhaps can be more on their terms, on their timeframes, and with their opinions.

Completely subjective take by you with similar epistemology around value as the blog author.

People might not care. I might identify as a runner because I bought a little jacket, expensive shoes, and wide-purple-tinted sunglasses, do I have to run? Not necessarily if the objects and my identity gives me the feeling of completion and satisfaction.

If your premise was true for all people, and the sense would be distorted, we would not see these phenomena, and people wouldn't listen or engage with AI-content. But the biological reality and the path of least resistance seems to prove us otherwise.

In this case, the user is deciding that they choose what progress is. I am saying, that people who use the tool and value the utility of it decide what is progress. If people listen to the podcast, or use doctors in the phone because it provides them any value, it will be a change and a perceived progress for them.

If the generated podcasts did not bring any value to the users, such as validation, or engagement, they would not use them, and there would be no change.

If you only care about the material and physical utility of the product, you can order the sweater from AliExpress for 5% of the cost and no time spent.

I am with you until you make this assumption:

But ultimately all these pleasant sensations aren't backed by a connection to the real. If you're going to talk about the history of knitting, shouldn't it be the real, evidenced history? As done by real (usually) women? Otherwise you're just knitting a pleasant fantasy for yourself.

If the real is the feeling you get from listening to the podcast or identifying with a subculture, then that is the real for that person. Factual, grounded information is just one take. If it was not this way, we would have much less myths, religions, etc historically.

People will feel the same degree of joy and completion when the final word of the podcast is read like you feel when you finish a really complex piece of work.