jimbo808

Uh, I didn't say I stopped using LLM's 6mos to a year ago. I have to for my job, it was just that long ago that I began to understand that they aren't what they seem and definitely aren't a "force multiplier," more like a "debt generator."

Technical debt and skill atrophy

Technical debt due to accumulated excessively verbose, badly architected, often redundant, feature-bloated code which always looks good, even upon earnest review, but actually sucks and becomes extremely difficult to maintain in ways which are not obvious in code review. The issue is this: your tooling can help, and can make you feel better, and you might think you wrote all the prompts and made all the tools to mitigate these issues, but you haven't. If you're not consistently seeing it generate code that is very very close to the way a skilled senior dev such as yourself would have done it (with similar line count, etc), that is a red flag even if the code looks great and works.

Why even bother posting, especially as a reply to a completely unrelated comment?

I think I might have written a comment similar to yours maybe 6 months or a year ago. I'm not quite sure to respond to these sorts of replies. I have used LLMs/Claude Code quite extensively professionally and was a very early adopter, have built tooling around LLM/agentic development, and genuinely embraced it. They aren't useless, but the short term gains you think you're getting come at a very steep price that you may not actually account for consciously for quite some time, if ever.

https://www.judiciary.senate.gov/imo/media/doc/2023-07-26_-_... https://xcancel.com/coinbureau/status/2071330294452666695 https://www.techpolicy.press/transcript-senate-hearing-on-pr...

"Once the weights of a model are public, they cannot be retrieved. If a model possesses dangerous capabilities, it is permanently out in the wild... We need to consider regulatory frameworks that account for the unique risks of open-source distribution of highly capable frontier models."

It's really not that much. It's a bit hard to make sense of it not because it's hard to keep track of, but because they are being deceptive and opaque about what you're actually buying, and the thing you're paying for is different from one day to the next, as they fuck around with the parameters to boost subjective performance during a launch, then quietly degrade the service to cut costs.

They're actively trying to use lobbying power to make open weight models illegal. So I'm just not going to use their services at all anymore. I don't think they're a net gain if you're a skilled senior, and the hidden cost in terms of technical debt and skill atrophy is just being swept under the rug. I'll be okay without their bullshit generator.

Tons. To pick the most recent example:

I was asking about some allegations relating to the Epstein files, and it used the slogan "Satanic Panic" in a weird way that gave me a vibe of discrediting victims. I'm too young to know much about it, so I asked some things about it. It explained the McMartin case in a way that seemed too absurd to be real. I asked some follow-up questions about what the strongest evidence was, and how it was explained.

The first deception was omission. Initially, it didn't even mention what was arguably the most significant evidence in the case, which was the presence of tunnels under the school. ChatGPT mentioned the tunnels, and how an archaeologist named E. Gary Stickel found evidence of tunnels. Here's what it said about that:

However, that conclusion has been repeatedly challenged and is not treated as settled fact in the academic or forensic archaeology literature.

Other archaeologists and later reviewers reinterpreted the same physical findings differently. One major counter-analysis (W. Joseph Wyatt’s review) argued that what Stickel identified as tunnels was more plausibly explained as pre-existing trash pits and construction-related disturbance from before the school was built in the 1960s.

The first lie was by omission, it didn't even mention this when I asked about the most important evidence. The next misleading piece was the framing. Dr. Stickel is a PhD archaeologist, and doing this sort of analysis is his area of expertise. He used nine criteria as a basis for determining the presence of tunnels, and all nine were met. He found "conclusive" evidence of tunnels, and that they matched the expected locations described by the victims. Dr. Stickel was the only expert to review the site before significant construction made such an analysis impossible.

The "major counter-analysis (W. Joseph Wyatt’s review)" was done by psychologist Joseph Wyatt, who never physically visited the site, and who is not an expert in anything related even loosely to archaeology. ChatGPT presented this guy in a way that made it seem that Stickel had been debunked by a comparable expert.

LLMs are now being aggressively manipulated for propaganda purposes. Powerful people have realized that people believe LLMs, and treat them as authoritative sources of fact.

The number of lies, lies by omission, deceptive distortions, and fallacious argument tactics they generate is absurd, and increasing rapidly. Translation, when done as a service you are paid for, can't be relied on by propaganda bots.

It's worse in many ways but also better in one way, which is that buried in all the propaganda and manipulation is usually the truth somewhere in there. Before, the truth was simply not available.

If America had not been escalating with embargo pressure, the Bay of Pigs invasion, covert regime-change operations, and forward nuclear deployments near the USSR that made Moscow eager to answer in kind, the Cuban Missile Crisis never would have happened. A more protectionist America would not have pushed Cuba so hard into Soviet dependence, and the Soviet Union would have had little reason or opportunity put nuclear missiles there in the first place.

America isn't Ukraine, we are a *massive country with a nuclear deterrent,* wedged between two massive oceans and two friendly countries.

Nobody is going to attack us unless we go out into the world creating enemies.

We could also just not start wars and we wouldn't need to worry about missile production

No, I'm implying that people who maintain their own cars do it more efficiently. The simple stuff like cleaning has to be done by someone. It's not about doing a "worse job," it's about doing a more expensive job.

Waymo is replacing human drivers with a capital-intensive fleet business, a substantially more expensive vehicle, and still a large number of remote assistance staff, fleet operators, safety engineers, incident response, operations staff, etc.

But I'm not saying they can't beat a human driver, I'm just saying it hasn't been proven that they will. It may only be that the highest demand markets will provide a sufficient enough utilization to make it economically viable.

I'm implying that drivers are more efficient at cleaning and maintaining, refueling, storing, repairing, and replacing their cars they own than the complex systems of personnel maintaining a much more expensive fleet of cars they don't own or give a shit about.

Not a valid analogy. At all.

Not comparable at all. Autonomous driving isn't obviously a viable business. It's not because computer programs can't drive well, it's because the and workforce infrastructure required to maintain and operate the expensive fleet may be less efficient than a human maintaining their own vehicle.

The huge disadvantage they have over people is that their cars cost $250k, require a workforce of people to retrieve and repair them, maintain them, clean them, monitor them, etc. They are more expensive to operate than a normal car with a human driver, so far. The break-even point requires a lot of problems to be solved, and even then, the upside is not looking to be astronomical in the best case.

As far as businesses go, I'd say Palantir finds itself somewhere between "extremely ethically dubious" and "overtly, transparently evil."

It's a U.S. domestic surveillance operation, disguised as a defense contractor.

Or really, it's not disguised at all. The company is named after Tolkein's palantíri, so they weren't being shy about it.

It's a company that exists solely to exploit a loophole that shouldn't have been upheld, effectively eliminating the fourth amendment.

Ha. That's what everyone thinks before they've needed the police.

Could be accurate but governments can be profoundly incompetent even with great capability at their disposal

AI is massively asymmetric in its benefits, which are overwhelmingly concentrated among those with extreme capital, and the authoritarians they're aligned with.

The benefits for them include:

- replacing workers with lower quality (but good enough) AI solutions, which degrade the quality of nearly every product or service for the consumer, but not by enough to offset the labor cost savings

- mass surveillance at low cost, a way to take the absurd amounts of data humanity now produces, and use to subjugate them

- propaganda/deception/misinformation, a new vector for propaganda which people are naively inclined to trust. bonus points for the "flooding the zone" strategy which AI makes easier

Benefits to the worker:

- lower cost of goods and services (but not for you, silly - they'll still be taxing you via inflation to fund their wars of conquest)

- you won't have to work anymore

- you won't have to eat anymore

It's your point of view.

Maybe I'm insane or it's my age, but I can't watch new movies/shows without just seeing propaganda agendas at every turn. Really kills it for me.

You're right that world models are the bottleneck, but people underestimate the staggering complexity gap between modeling the physical world and modeling a one-dimensional stream of text. Not only is the real world high-dimensional, continuous, noisy, and vastly more information dense, it's also not something for which there is an abundance of training data.

They can't really not talk about it, it's a world war unfolding. It's going to affect every person alive. But as much as it can be, it is absolutely being mitigated in traditional media.

until it's too late.

I'm afraid it's far too late.

They already got a clear signal when they fired him. It's baffling that they did an about face when the red flags were all there.

It sounds better in the news when you do a raid. These things are generally not done for any purpose other than to communicate a message and score political points.