sponaugle

The best part of this workflow - which I see often - is that by having someone build custom software to automate some process they often step back away from the process being their job. That eventually translates into them understanding that some (or sometimes most or all) of that process is not needed. There are so many corporate processes that were implemented and then become the way... and if there are people who identify that process as being their job those people resist attempts to optimize that process.

I have seem several people use AI to write apps to automate a process and along they way finally ask the question 'do we even need this process?'.

Regrettably this does not happen everywhere.

What I was commenting on was the concept that a small model at home is somehow more efficient. To make a reasonable and fair comparison you would compare many people running a small model at home vs those same people using what would likely be a shared resource in a datacenter.

The core concept is that tokens/watt is tokens/watt ( for a given model of course ). A computer at home is actually less efficient overall because most of the time it is not doing tokens but still using a small footprint of power.

The revenue pressure is an interesting problem , but I suspect the actual demand math will be much more complicated.

I find local models interesting for sure, and run several on my own personal DGX cluster. I am however most certainly not power efficient!

"But there’s another challenge: local LLMs. It’s already possible to run LLMs on local hardware, and that’s only going to get easier in the future. Apple’s M-series chips are extremely good at doing this today. Open weight (read: free) models are widely available and good enough that most people probably couldn’t tell the difference. They also have the benefits of running on hardware that’s sipping power most of the time, rather than slurping it down in massive data centres."

This is such an odd and illogical conclusion. If a smaller model can be sufficient (which is not something I would have said), that smaller model can be ran in a datacenter. The idea that a small model running at home is 'sipping' while that same small model in a datacenter is 'slurping' is absurd. The datacenter will have much greater overall efficiency in both power usage and total cost to implement. Of course if you compare a small home model to a DC frontier model the power usage is different, but so is the output.

That brings back memories! Riding in the Transport to pizza hut with tanks of refrigerant banging around in the back. I remember when Ardent appeared! It was such a cool machine, and to this day I occasionally scan e-bay to see if one comes up.

Indeed what made George special was that he was a broad 'engineer' first, and a specialist later. Everything was an engineering problem that could be solved. Perhaps most of all he believed in the students, hiring as many as he could get approved.

I found a 'bug' in the debugger on the NP1s that gave me a root escalation, and when he found out he said I should come work for him and that was that! About a year in we scrapped another older system and he had a spare disk that we put in the EE NP1 (en.ecn.purdue.edu) and mounted it as '/hogs' because I was always hogging disk space with stuff I downloaded. Funny enough I still have an exabyte tape backup of that drive.

Great times for sure. RIP.

That is awesome. The NP-1s were great. I spent lots of time working on en.ecn.purdue.edu - Some tape drivers, some maintenance, and lots of software projects - it was really cool that in those days everyone was on the same machine, working from terminals. Good times!

George was really into video stuff - he had stacks of 8mm video tapes in his office, and of course stacks of exabyte drives. He had many different cameras and was always trying new ones out. He was also a really early adopter of laserdiscs, and I have a few discs he gave me when I graduated.

I was holding the camera for some of these videos. Such a great time!

Sad to hear! I worked for George for all of my undergraduate time at Purdue. He was an amazing boss with such a passion for all things unix. For a while he had the UNIX license plate on his minivan.

I am similar in that all of my interactions are with my real name and it is unique enough that just putting it into google will instantly identify me. There is one other 'jeff sponaugle' but I think he is far more annoyed with my presence than I would be with him.

On the plus side, someone will sometimes say while talking to me - oh your are that Subaru guy, or that youtube guy, or whatever and that is fun connection.

A fantastic read, and really interesting to see the use of Forth. I remember Forth having a bit of popularity in the 80s. This was such an amazing game, especially in that you felt like the world was huge with the encouragement to just explore.

The other game this reminds me of is a game for the TI99/4a called Tunnels of Doom. It was a cartridge game that also had a floppy or cassette data load. It had a dynamic dungeon creation so every time you played the game you got a new unique experience. That would be an equally challenging one to reverse engineer due to the oddity of the GROM/GPL architecture in the TI99/4a.

The study looks at a wide range of different tests spanning many different areas of expertise and output types. Some of the tests, like the web vis tasks used Sonnet not Opus (which was not out at the time). It is similar to testing a car to do many different things, but only one of the tests is the actual driving somehwhere and many of the others are based of the fabric used in the interior. This gives a very broad "96% failure" while missing the observation of the successes. Of course AI can't do everything, and nor can I.

One of the most interesting observations about AI is the timescale at which the favorite model and favorite task changes. Before November I found Sonnet to be interesting, but not moving that much of the needle. Once Opus came out it was clear the needle was not only moving, but moving fast.

Yea, that makes sense as you would need quite a bit of information across a reasonable temporal range if the identifying qualities are movement related. Very interesting.

This is a VERY controlled environment - and they used 20 passes of each person walking with direct knowledge of each person to train for identity. They did no tests with multiple people walking at the same time, or with any other external moving distortion effects (doors opening, etc) . This is very far from actual 'identification' of people in real public settings - and no doubt the cell phone everyone is carrying with them offers many orders of magnitude better opportunity. In a real crowded environment this would be nearly worthless.

The devices that reported BFI information were also stationary, and there were no extra devices transmitting information that would be conflicting.

A single camera would be much more effective.

Oh they are both loud and eat a ton of power. I think the 9305 is at least 800 watts at idle. That is the biggest downside of the retired datacenter gear... yo u really need a dedicated room with power, cooling, and sound isolation.

Agreed - the big thing is 100g is much much cheaper now as so much 100g gear is coming out of datacenters. So many of those older ConnectX4s and 5s, plus lots of switches and optics. 100g really is the new 10g for homelabs.

I'm using ConnectX5s for most, and some ConnectX4s in the older servers. Both of those cards have really come down in price in the used/ebay market. I have been playing around with some different optics - I have a bunch of CWDM4s which are very inexpensive and use a single SingleMode pair.... but of course they run hot so if you have them in servers without good air flow you might have problems.

I'm using mostly fiber just because the servers are connected to Cisco 9305 with 72 100g ports.

Yea! Those MS-O2s look great, so I may need to upgrade! I did get a couple of DGX Sparks to play with.

Yea I'll dig up the link I used - there was a great reference about getting the thunderbolt working after reboots/etc.

I have a 3 node proxmox setup on MS-01s using a 25G Thunderbolt ring for Ceph, and indeed it took a lot of hoops to get it working correctly and reliably. I did manage to get it such that nodes can go up and down without needing to unplug anything, and the dynamic routing works if a node disappears. Performance is pretty good, with a more realistic 20ish gbit/sec.

"because I can, and it is fun." The best answer! I am most of the way done with upgrading most of my homelab to 100G from 10G, but there really isn't a practical reason for it. 100G has dropped in price so much as datacenters are all about 400/800G now.

I think understanding stellar processes and then using that understanding to theorize about other observations is a skill. My point was that observing can be a fantastic way to build a skill.. not all skills, but certainly some skills. Learning itself is as much an observation as a practice.

Agreed - There is no guarantee of what will happen in the future. I'm not for or against the outcome, but certainly curious to see what it is.

"it lost its value".

It has not lost its value yet, but the future will shift that value. All of the past experience you have is an asset for you to move with that shift. The problem will not be you losing value, it will be you not following where the value goes.

It might be a bit more difficult to love where the shift goes, but that is no different than loving being a artist which often shares a bed with loving being poor. What will make you happier?

"No-one has ever learned skill just by reading/observing" - Except of course all of those people in Cosmology who, you know, observe.

"If humans are machines, they are still a subset of machines and they (among other animals) are the only ones who can be demotivated and so it is still a mistake to assume an entirely different kind of machine would have those properties."

Yet.

"I think 2026 is the year of Business Analysts who were unable to code." This is interesting - I have seen far more BAs losing jobs as a result of the 'work' they did being replaced by tools (both AI and AI-generated). I logically see the connection from AI tools giving BAs far more direct ability to produce something, but I don't see it actually happening. It is possible it is too early in the AI curve for the quality of a BA built product to be sufficient. CC and Opus45 are relatively new.

It could also be BAs being lazy and not jumping ahead of the train that is coming towards them. It feels like in this race the engineer who is willing to learn business will still have an advantage over the business person who learns tech. At least for a little while.

Oh yea, 100% agree.

On that avenue, I do push hot air from my homelab into my upper garage for heat. If it below 50deg outside I also bring in some cold air from outside. Both are somewhat free offsets for heating/cooling.

I used about 64MWh last year, not counting what I used for EV charging (Which is on a separate meter). I also produced about 20MWh from Solar. With the EVs I would guess the total is around 70MWh.

Some of this extra is certainly my 6kw homelab + HVAC for that. ;)

I have met more other founders that came from large engineering state schools then the next tier-up of prestigious schools, with one exception being Stanford. Of course these larger schools produce far more students. Purdue (where I went) generates over 3000 new engineering graduates every year.

From the hiring perspective - A degree from from ivy school means so little compared to the actual skill level of the applicant. I honestly could not tell you where the last 10 people I hired went to school, or if they even did.