The Last IT Guy

https://suthakamal.substack.com/p/the-last-it-guy

by suthakamal • 4 months ago

2 3 4 months ago

suthakamal.substack.com

suthakamal• 4 months ago

AWS is Windows NT, and Agents don't need managed service

jfalcon2• 4 months ago

They do need hosting and now they need a very particular hosting with very particular hardware which is the bottleneck.

Now here is the trick - exporting the magic that makes LLM work (transformers) into ASIC hardware to get it out of the GPU. The problem being the blackbox of logic gates within the gpu that makes the LLM work.

There are a few that have figured it out. There should be more, way more. Else this will never scale and we'll be stuck within the trap of cloud - because nobody is asking for less except in their bills.

jfalcon2• 4 months ago

One of the things I found interesting was that you were contrasting the two sides as either/or. Like it were this meme: https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcTz2_fj...

Memes are funny when they hold an element of truth to them. And reading through your blog post, there are some moments of laughter.

But, your conclusion that it's just about one side coming to the other rings hollow.

The reason you are fixing a database at 9pm on a Friday is because you or your predecessors or company in general did not care to pay off their tech debt. Doing int and bigint was always a bug to bite you in the ass. Not having your cloud metrics properly setup and observable is also tech debt because there was a decision made somewhere not to build it, or document it in a runbook, or whatever. And there you are, 9pm on a Friday, self-doubting yourself.

Agentic AI, I hope, will evolve into a decentralized model - because the user is the edge, not the cloud. The problem is that the hardware that it takes to run LLM at scale only works in the cloud - until the blackbox of LLM magic is distilled into an ASIC well enough to be mass manufactured.

Without this happening, without a way to take a state of the machine and put it into soda cans, it remains the realm of those with the cash to keep their well from drying out.

The Last IT Guy

Related Stories