mechagodzilla

"Household Debt Service Payments as a Percent of Disposable Personal Income": https://fred.stlouisfed.org/series/TDSP

Relatively accurate (I think they're pretty close like 70%+ of the time). Significantly better than just projecting that the next five years will be like the last 5 years.

You haven't really been getting 'human written and thoughtful content' for a vast swath of search topics for probably 15-20 years now. You get SEO-hyper-optimized (probably LLM-generated for anything in the last 3 years) blog spam. In terms of searching for information and getting that information, there are a lot of topics where an LLM-generated result is vastly better just by virtue of not being buried inside blog spam. The slop ship sailed years ago.

It's not really an apples-to-apples comparison - I enjoy playing around with LLMs, running different models, etc, and I place a relatively high premium on privacy. The computer itself was $2k about two years ago (and my employer reimbursed me for it), and 99% of my usage is for research questions which have relatively high output per input token. Using one for a coding assistant seems like it can run through a very high number of tokens with relatively few of them actually being used for anything. If I wanted a real-time coding assistant, I would probably be using something that fit in the 24GB of VRAM and would have very different cost/performance tradeoffs.

I've been running the 'frontier' open-weight LLMs (mainly deepseek r1/v3) at home, and I find that they're best for asynchronous interactions. Give it a prompt and come back in 30-45 minutes to read the response. I've been running on a dual-socket 36-core Xeon with 768GB of RAM and it typically gets 1-2 tokens/sec. Great for research questions or coding prompts, not great for text auto-complete while programming.

1-2 tokens/sec is perfectly fine for 'asynchronous' queries, and the open-weight models are pretty close to frontier-quality (maybe a few months behind?). I frequently use it for a variety of research topics, doing feasibility studies for wacky ideas, some prototypy coding tasks. I usually give it a prompt and come back half an hour later to see the results (although the thinking traces are sufficiently entertaining that sometimes it's fun to just read as it comes out). Being able to see the full thinking traces (and pause and alter/correct them if needed) is one of my favorite aspects of being able to run these models locally. The thinking traces are frequently just as or more useful than the final outputs.

You can keep scaling down! I spent $2k on an old dual-socket xeon workstation with 768GB of RAM - I can run Deepseek-R1 at ~1-2 tokens/sec.

Kodak didn't really have the option to compete. Their business was largely film, which just disappeared completely, and even digital cameras got replaced pretty quickly with phones. There was nothing to pivot too for Kodak.

Did you have to do anything special to get the SSD to play nice with OS9? I tried adding one to a 300MHz G3 iMac and it took forever to initialize on boot and would randomly stall a lot.

If anyone can pay-as-you-go use a fully automated factory, and the factories are interchangeable, it seems like the value of capital is nearly zero in your envisioned future. Anyone with an idea for soup can start producing it with world class efficiency, prices for consumers should be low and variety should be sky-high.

Buy a used workstation with 512GB of DDR4 RAM. It will probably cost like $1-1.5k, and be able to run a Q4 version of the full deepseek 671B models. I have a similar setup with dual-socket 18 core Xeons (and 768GB of RAM, so it cost about $2k), and can get about 1.5 tokens/sec on those models. Being able to see the full thinking trace on the R1 models is awesome compared to the OpenAI models.

Yeah, it was just a giant HP workstation - I currently have 3 graphics cards in it (but only 40GB total of VRAM, so not very useful for deepseek models).

I use a dual-socket 18-core (so 36 total) xeon with 768GB of DDR4, and get about 1.5-2 tokens/sec with a 4-bit quantized version of the full deepseek models. It really is wild to be able to run a model like that at home.

Interns and new grads have always been a net-negative productivity-wise in my experience, it's just that eventually (after a small number of months/years) they turn into extremely productive more-senior employees. And interns and new grads can use AI too. This feels like asking "Why hire junior programmers now that we have compilers? We don't need people to write boring assembly anymore." If AI was genuinely a big productivity enhancer, we would just convert that into more software/features/optimizations/etc, just like people have been doing with productivity improvements in computers and software for the last 75 years.

I have a $2k used dual-socket xeon with 768GB of DDR4 - It runs at about 1.5 tokens/sec for the 4-bit quantized version.

I've done all of those except tend livestock and build a house, but I could probably figure those out with some effort.

So applying this to China and the USA - the USA has a median household income of ~$80k, and China, in terms of purchasing power parity, has a median household income of $32k. China has a workforce of ~775M people vs 163M people in the USA. The USA has ~7M unemployed people. For the USA to stop importing goods from China, we would need to 1) stop consuming those goods altogether (even if they're intermediate goods that go into things we do manufacture), or 2) employ US workers to make those goods instead of whatever they're currently doing (either employed or unemployed).

Employed Americans largely earn a lot more than employed chinese people, and there just aren't very many unemployed Americans. Based on the value of imports to the US from China vs Chinese GDP($440B vs $17.8T), we import about 2.5% of their output, or the equivalent of ~20M people's output if we naively scale. I don't think there is any way around the fact that significantly reducing imports from other countries means we significantly reduce our consumption in absolute terms (i.e. we have a poorer standard of living), just out of spite for other countries.

Ha! When I was first learning to program in high school, I wrote a 'distributed monkeys-on-typewriters' simulator. I somehow acquired a stack of surplus Pentium 100s that I had running in an unused closet at the school, communicating with each other over IPX. I remember the server had a fun 'Guess-operations-per-second' (GOPS) realtime display.

I think we can already get open-weight frontier class models today. I've run Deepseek R1 at home, and it's every bit as good as any of the ChatGPT models I can use at work.

That's been the 'endgame' of technology improvements since the industrial revolution - there are many industries that mechanized, replaced nearly their entire human workforce, and were never terribly profitable. Consider farming - in developed countries, they really did replace like 98% of the workforce with machines. For every farm that did so, so did all of their competitors, and the increased productivity caused the price of their crops to fall. Cheap food for everyone, but no windfall for farmers.

If machines can easily replace all of your workers, that means other people's machines can also replace your workers.

None of that means that the current companies will be profitable or that their valuations are anywhere close to justified though. The future could easily be "Open-weight models are moderately useful for some niches, no-name cloud providers charge slightly higher than the cost of electricity to use them at low profit margins".

After being a professional programmer for ~20 years, and recently playing around with leetcode - my main issue with leetcode is that there's almost no overlap between leetcode problems and the problems I actually encounter in the wild. The validation tests often have silly corner cases that force you into a single answer to avoid timing out. It's frequently as much work to understand what the problem is actually asking you as it is to implement a solution. Just like I've found ChatGPT to be pretty mediocre at writing the sort of code I work on, but others swear by it, maybe some peoples' dayjob actually looks like writing leetcode all day? I know a lot of interviewers use it, but it feels so disconnected from actual engineering work.

I married at 22 and moved to nyc. I lived through meeting peers that thought we were freaks (“you two are definitely gonna get divorced”), to the gradual normalization as we got older and more of our friends married. I knew lots of people that transitioned from young and invincible and definitely never going to get married to dismayed at coming up on 40 and feeling like they hadn’t quite started “real life” yet. I don’t like the article’s depiction of the guy “taking her under his wing” had they gotten married earlier, but I will say that there is something great about sharing your youth with someone unequivocally on your side, if life gives you the opportunity.

But the open models create a (rapidly rising!) 'floor' - models that are worse/less capable than the best open-weight models effectively have zero economic value to their creators, even if they just spent $10B to create them. What fraction of queries require a better-than-R1 answer? Every time that floor rises, it puts even more pressure on whatever narrow, temporary advantage these closed companies can achieve to actually justify the valuations.

And the Llama series, and they have half a dozen closed competitors with similar performance/capability.

It really seems impossible to justify these evaluations from any kind of economic perspective.

I think that's the right interpretation, but that's pretty weak for a company that's nominally worth $150B but is currently bleeding money at a crazy clip. "We spent years and billions of dollars to come up with something that's 1) very expensive, and 2) possibly better under some circumstances than some of the alternatives." There are basically free, equally good competitors to all of their products, and pretty much any company that can scrape together enough dollars and GPUs to compete in this space manages to 'leapfrog' the other half dozen or so competitors for a few weeks until someone else does it again.

If you don't need to pay for the model development costs, I think running inference will just be driven down to the underlying cloud computing costs. The actual requirement to passably (~4-bit quantization) run Deepseek v3/r1 at home is really just having 512GB or so of RAM - I bought a used dual-socket xeon for $2k that has 768GB of RAM, and can run Deepseek R1 at 1-1.5 tokens/sec, which is perfectly usable for "ask a complicated question, come back an hour or so later and check on the result".

Or, like Meta, they make their money elsewhere and just seem interested in wrecking the economics of LLMs. As soon as an open-weight model is released, it basically sets a global floor that says "Models with similar or worse performance effectively have zero value," and that floor has been rising incredibly quickly. I'd be surprised if the vast, vast majority of queries ChatGPT gets couldn't get equivalently good results from llama3/deepseek/qwen/mistral models, even for those paying for the pro versions.

It does seem like it will be very, very hard for the companies training their own models to recoup their investment when the capabilities of open-weight models catch up so quickly - general purpose LLMs just seem destined to be a cheap commodity.

The $500 going to the "AI Owner" instead of labor (i.e. the human lawyer) is the productivity gain though, right? And if that was such a productivity gain (i.e. the marginal cost was basically 0 to the AI owner, instead of, say, $499 in electricity and hardware), the usual outcome is that the cost for such a product/service basically gets driven to 0, and the benefit from productivity actually gets distributed to the clients that would have paid the lawyer (who suddenly get much cheaper legal services), rather than the owner of the 'AI lawyer.'

We seem pretty likely to be headed towards a future where AI-provided services have almost no value/pricing power, and just become super low margin businesses. Look at all of the nearly-identical 'frontier' LLMs right now, for a great example.