sfink

At least Opus up to 4.7 or so, my experience is that Claude often uses "yes, with caveats" in place of "no, you numbskull".

"Is a meter the same as a foot?"

"Yes, exactly, you have it now, except they're different distances."

Maybe it's because I ask it to quiz me, and it really doesn't like to tell me I'm wrong. I also got a fair amount of

Claude: Ok, I will test your understanding. Question A? Question B? Question C? Question D?

Me: A=10. B=2. C=121. D is not solvable.

Claude: You got most of them right! You're very astute in saying that A=10, but actually it's 7. B=2 is exactly right! C could be 121 if we were talking base 4, but we're actually in base 10 so it's 25. D is trivially solvable and is 0.

Me: ...isn't that like 1 out of 4? How is that "most of them right"?

Claude: You're absolutely right! ...blah blah blah

At Mozilla, we had a set of whiteboard tags we could set on bugs, like "[crash]" or "[compat]" or "[leave-open]". That last was used when there were multiple patches attached to the bug, and we wanted to land only some of them without automation closing the bug once they landed. (It's common to have alternate approaches or test cases also attached to the bug, so you normally don't want to wait for all of them to land before closing the bug.)

I started using "[leave-open" for those.

It lasted for a couple of years, until someone went through and "fixed" them all.

Just be thankful that we don't share the penchant for giving credit to discoverers. Imagine calling a cache a "Murphy/Steinman/Sokolov structure" (made-up names).

I mean, we do for some things, especially algorithms (Boyer-Moore). Probably for the same reason the mathematicians do -- there aren't readily available real-world analogies.

And I won't even mention the branded future, with its "Google HyperZipper String Search" and "OpenAI/Red Bull speedmaxx distributed consensus algorithm"...

For better or worse millions of Americans voted for the guy doing the deportations.

Uh, the idea of democracy is not that voting reveals universal preferences or something. It is ok to disagree with whoever gets elected, and continue working towards an alternative you believe to be better. In fact, democracy depends on that; otherwise, why not have a single election with no term limits? There is supposed to be ongoing difference of opinion.

I also find it difficult to reconcile not using AI for weapons.

You can't reduce "AI for autonomous lethal weapons" to "AI for weapons".

The main answer would probably be: because he worked for Google Deep Mind as opposed to Google in general, so the killerbot and mass profiling concerns were immediately relevant to him and his work.

Your comment seems to be whataboutism. The article was not an essay trying to prove that he knows the secret to being maximally effective at politics, or whatever it seems you were expecting?

The weaknesses can be multiplicative as well. One division bleeding capex can drag down all the rest, no matter how well they might be doing. And the P/E ratio on all of them is riding unrealistic expectations, which can actually be fine for a long time but forces growth even in areas where it doesn't make sense. (Maybe that's where the "let's build data centers in a high radiation hard vacuum!" nonsense comes in; you just need a story of how the P/E ratio is possible to justify in the future? No need to argue over likelihood, just have a tale to tell?)

Prohibiting a user of your software from modifying it and using it to run their business goes against both freedoms 0 and 1.

Then it's good that it allows both modification and using it to run a business?

This “users are obligated ‘give back’ the moment they make money with the gift they were given” is nonsense

But they are under no such obligation! They can make all the money they want and give nothing back. They can even modify the software to better serve their business. The only restriction is that if they do so, they have to make their modifications available. Which means they're way ahead of where they were before being given the initial software; why do you feel a software developer who decides to give the world a gift should be restricted in what gift they're giving? "Thanks for the chocolate, but the bar was too small so I didn't have enough left over for my kid to try some. Why do you hate my kid?"

A business making money using free software doesn’t take anything away from the releasing organization.

First, that is false. They could damage the market for the original software. (And if they don't modify the software, then there's no problem in the first place.)

Second, why are you so hung up on the "making money" part, when that is explicitly allowed by the AGPL? It's just kind of bizarre -- it's a license that says over and over that you can charge for everything related to it, and you're complaining about it being hostile to people who want to charge money for things.

Thinking about it, I'm wondering if this is genuine confusion and you don't know what the AGPL is? If so, maybe start by searching for "charge" in https://www.gnu.org/licenses/agpl-3.0.en.html . It has nothing against "using it to run [a] business". There is no "moment they make money with the gift" that changes anything: you are explicitly allowed to charge for anything you like -- distribution, usage of the service, support, whatever.

they failed to address that these labs are creating a public good as well. Who's to say that this "theft" is larger than the production of public goods that these labs give to the public in the first place.

Those aren't public goods, they're private goods. The difference is already apparent, and will become more so over time. But yes, they are "goods"; they have a lot of value.

I'm less interested in punishing the past theft that enabled these goods, and more worried about the ongoing damage they are doing to the ecosystem that birthed them. Good artists copy, great artists steal, but if a community is wholly dominated by these "great artists" then it will not survive as a community and will cease to produce anything worth copying let alone stealing.

It's been said a thousand times but the ability to produce copyrighted material is not copyright infringement.

Aside from fair use, yes it is.

If I draw The Simpsons on a piece of paper - whether or not I used AI to create it - it's not copyright infringement.

Only because of the fair use doctrine, which is limited. If it damages the market for the original, then a court could definitely declare it to be infringement.

Copyright infringement is if I tried to sell that Simpsons work as my own, putting up for consumption and reaping the monetary benefits.

No. Copyright infringement requires neither sale nor misrepresenting a work as one's one. It also covers derivative works, not just perfect duplicates. Copyright covers reproduction and derivate works and distribution/performance -- you can get in trouble for just one of those. Taken strictly, that would be a horrible world, but fortunately the fair use doctrine weakens those quite a bit. On the flip side, if AI reproduction destroys the original markets -- as it is quite obviously doing right now -- then it's going to have a reckoning at some point, given how its legal status is completely based upon getting a free ride by claiming fair use.

That's pretty much the reaction I had, except that I do find the argument compelling that AI is damaging the ecosystem it feeds upon.

I think Google's AI results are probably the prime example here. Those results are often quite good, and starve off the visits and hence revenue stream of the sites where the results are sourced from. Additionally, there's no way to robustly attribute that information anymore either; those links were already broken pre-AI by freeloading aggregators. So potential information producers can't afford to host their valuable information since they will pay proportionally to their information's value (as translated into bandwidth).

But it's tricky. If you charge AI companies proportional to the damage they do, then you need to assess that damage and you'll be caught in a no-win cat & mouse game where the AI companies outsource the damage and you try to track it back to them. They win if they bump their revenue, but they also win if they conceal the damage. (Just like companies can use shell companies, bankruptcy, and asset-only purchases to avoid Superfund responsibility.) If you charge proportional to revenue, then there aren't conflicting incentives; companies win by increasing revenue full stop. But I do agree that this shouldn't just apply to the big companies; distillation / model extraction (adversarial or not) should not be a way to avoid fees.

Though what really seemed off to me about the proposed solution was the use of the fees (to pay Americans, no less!) Perhaps upcoming essays will justify this more, but to me it seems like those fees should be applied directly to the health of the ecosystem. It should be used to fight pollution, in this case AI slop overtaking the Web. It should support the value creators, eg open source developers being overwhelmed by the AI tsunami. It should go towards serving and moderating online communities that create the very value that the LLMs are trained from. In theory, paying a whole bunch of Americans a trickle of blood money will end up going towards these purposes, but I'm very skeptical: first, the benefit will be very diluted. Second, it's more likely that the excess money will end up in Google or Amazon's pockets. The whole system is already set up to route any advantage to the big players. The payments are at least as likely to damage the ecosystem as they are to support it. They would just feed the capitalistic wolf and further reinforce the setup where monetizers win and value creators lose. If you could magically route the money to value creators, that'd be great. But you can't.

It'd be ok to pour money into a bucket if it had a small hole or two, but pouring it faster into the sieve we have now is not going to help anything.

That's a fascinating link, and it sounds like what I'm talking about is what's called "engineer's blue" in the link.

But I think it would be vastly more difficult to grind two massive stone blocks against each other than to just ram one against the other. Not unless you stacked them, anyway, and if you stacked them I'm not sure if you could move the top one side to side in order to do the grinding. Maybe with some kind of grit, I don't know. Still seems harder.

Also, grinding methods end up removing more material (bad for teeth!), and I would expect more overall physical work to be done in order to remove that material (bad for massively heavy stone blocks).

As for making them flat, that seems unnecessary to me. But then, I'm not a pharaoh. (Even for a pharaoh, it seems like only the seems would need to be straight. Nobody could tell about the faces after assembly.)

Then again, after some quick researching, it seems like there's a good chance that the well-fitting blocks (which are not all of them) may have been cast out of a concrete-like slurry, not hewn.

(I know nothing about this subject, feel free to ignore me.)

My dentist is pretty good at doing this too, by putting marking paper between my teeth and having me bite down. I wonder if a similar technique could be used:

Have the blocks close together, constrained to only move on a single axis by rails or whatever. Drape a thin sheet of material over one of the blocks, the non-moving one (perhaps it's an already-placed one?) Maybe it's something that visibly shows when it's crushed, or maybe it's coated with the blood of the powerless. Smash the other block into it. Pull them apart and look where they made contact. If it's mostly everywhere, done. If not, grind down or chip out the parts that touched. Repeat until you run out of innocents.

To do the very last block, you'd have to meld two sides, remove a block, fix up the other side, and then put it back in. Which might make this testable.

But I'm just pulling stuff out of my nether orifice.

If it were the FBI, it wouldn't be "To Kill a Mockingbird", it would be "Amateur Forgery, volume XVII: Passports" or something. Well, or something similar that wasn't already illegal.

I really don't see the issue with local entities like a school board having some say in material that is available in a school.

Then you prefer a low-trust environment. I prefer a high-trust environment. A librarian shouldn't be putting 50 Shades of Grey on a grade school shelf to begin with. If they are, then you should be replacing the librarian, not micromanaging them. Book selection is their job. Let them do their job or don't; don't allow them the authority to only do half their job.

But a library acting like they are doing some brave act of resistance by putting out a stack of books that are widely available...and saying they are "banned books, this is banned books week, look at all the books that have been banned!" when really they are books that a school board in wisconsin said shouldn't be in an elementary school library because the sex scenes are not appropriate for 7 year olds seems really silly to me.

Again, that's a "replace or reprimand the librarian" problem. It's not meant to be a brave act of resistance, it's information to say "these books have been banned, look at them so you can better understand what books people want to ban and why". And obviously, it's more interesting than "these are books where the 3rd letter in the title is T" and so it garners more attention, but it's no more than that. If they're including one that was banned for dumb reasons as in your example, then that makes it a dumb display (and an inappropriate one if the display is also in a library for 7 year olds.)

Obviously, the OP is not the librarian, and is aiming for an act of resistance, so my argument mostly doesn't apply there. Though the part about choices having the potential of being dumb still does. The set of books that have been banned somewhere or other is quite large, it's not like it would have any meaning to have a display of all or even a random selection of them. That's a strawman. You're going to curate based on some metric.

I think the point was that it's difficult to notice in the first place, not that it would be hard to find once you know you're looking for something. You don't have a black WiFi router with antennae dangling down from the ceiling.

If you went the other direction and didn't worry about it being noticeable, it would be kind of a fun project to break up a book into a series of QR codes. A scavenger hunt, with each code's text ending with a clue of where to find the next?

You may be overthinking this. "Banned" in this case means that the usual person or people who choose what books to include are being overridden by a party with more clout. From the perspective of a school librarian, for example, book X has been banned. They no longer have the option of including it. (This is even true in the case where the librarian would not have included it anyway, for their own reasons.) They are prevented by the school board, an angry mob of parents, the state legislature, the FBI, or whoever. The fact that the public library down the road carries the book does not change whether that librarian has the option of including it in their school library's collection. They can't. They are banned from including it.

In general, because it's a flag that says to do things in an incorrect but faster way. It's like -ffast-math. The applications for which it's intended don't do anything where the incorrectness matters. Some random application falsely labeled hl2.exe may or may not.

What it should do is ensure some things not relevant to Half-Life 2 were not done, thus getting better performance for this game in particular, but there is no guarantee that same optimizations work for other applications or games, so one should not expect an overall improvement.

I can't quite parse this. Yes, there is no guarantee that the optimizations will work for another game, which is precisely why you can expect an improvement with hl2. With non-hl2, you may get an improvement, you may not, and you may get incorrect behavior.

Everything else is not the same, but hl2 doesn't use the stuff that's different.

I largely think that we engineers are to blame for LoC being still perceived as an asset rather than a liability. We are proud of stuff we create, but it turns out that you can't describe how "big" something is without some metric, and so we fall back on the metric that is easiest to compute.

Suggestion: we should all shift our terminology, and in particular make heavy use of phrase "...and it cost N lines of code". And say what we spent those LoC on.

"I implemented new feature X, and it only cost 200 lines!"

"That bug was brutal to figure out, but in the end it only cost 6 lines of code."

"It was doing something in case X that it didn't do in case Y, and it turns out that the distinction wasn't even needed. So I fixed the problem and saved 20 lines of code at the same time!"

Lines of code are a price you pay. We don't go around bragging about how we spent $200 without any mention of what we purchased with that money. Why do we do that with LoC? "I had to pay an extra $200 because I signed up late" and "I only paid $200 for my hand-painted artisanal pottery lamp hanger. Factory-made ones cost upward of $1200 on Amazon!" are two very different statements, and map to exactly the same distinction in code.

"RSS lies. Your process might not be using that memory. The allocator might be hoarding it."

Interesting writeup, but:

No, anonymous AI author. Your process is using that memory, for its allocator. Features like lower-latency allocations don't come for free, even when they turn out severely suboptimal for your particular case. Your code isn't using that memory, but a support library is. It is very much in your process.

Dammit. For a long time, I was blissfully clueless about AI writing style. But I recently read something that screamed en-SLOP, and now I can't unsee it anywhere. It hit me immediately with this article.

I want to go back to being dumb and naive. Give me the blue pill, please!

My post wasn't written in a way to make friends, but:

I've since rewritten nearly 100% of the prose in the analysis with my own, more inflammatory and verbose style. I also intentionally left in my natural mispellings and typos, to prove it was me.

Thank you thank you thank you. I would love to be able to describe how hard it was for me to think about the actual evidence you're presenting when reading about it through the AI writing, but I suspect it's one of those things where it bothers you or it doesn't. If you'd like to empathize, maybe I'll give it one try: imagine an otherwise solid PhD thesis written in crayon. The facts and evidence and reasoning are unaffected, but it's just so hard to take it seriously.

Anyway, with the rewrite I don't have to battle my kneejerk reactivity nearly as much.

I'm no expert like she is, but based on what I know, I agree with your wife on the statistics. That style of analysis is going to be the best you can do with the data available. It's an accepted way to stretch data without being too dependent on an assumed distribution. It's a good analysis. I still don't come away with the conclusion that concerns about AI code maintenance are necessarily overblown, but that's fine. I think your analysis project is a very solid contribution, and it's a hell of a lot more evidence-based than the rants people were posting.

Wow.

I am pretty insensitive to AI writing. I have never commented before about something sounding like AI, because mostly I don't notice. But this was so over the top that I spent the whole article trying to decide whether it was an intentional parody of AI writing style.

This article's language is not en-US. It's not en-BR. It's en-SLOP.

Yes, that was my clumsy attempt at AI parody. Here's another: this article doesn't just have AI tells. It is AI tells.

Every sentence is saturated with AI style. Perhaps the author so AI-indoctrinated that they can't see this? It doesn't read as even vaguely plausible human writing. Which is mightily ironic given the thesis of "AI generated stuff is just fine, m'kay?" The writing style does more to defeat its conclusion than the analysis itself.

As for the substance of the analysis, it seems pretty good to me but I see some flaws that weaken it a bit.

The presence of "The Outlier Nobody Noticed" proves nothing and deserves no more than a passing mention. A random release introduced way more bugs than the Claude-containing releases. That provides evidence that Claude doesn't introduce more bugs only if your hypothesis is a very naive "AI is the only thing that can ever increase bug introduction rates."

The whole analysis has very limited data. It's necessarily based off a single pair of releases at the very end of the chronological timeline. You would never be able to reject a null hypothesis based only on that, so it's even less sound to present it as proving the null hypothesis. (By the same token, it would be incorrect for critics to claim that it proves their point. Did anyone claim this, though? The heated complaints seemed more based on priors about AI code.)

"The critics' claim is a simple comparison: did the rate go up?" That's reductive. For one, these releases are known to be in reaction to a flood of (AI-discovered!) security reports, which is a novel situation and in fact is a huge confound to anyone arguing about what those two releases mean -- they're both heavily AI-written, but in response to an unusual situation. When the samples are only drawn from a distinct scenario, statistic analysis can only speak to the quality of code in that scenario.

Also, another reasonable hypothesis could be: AI-written code has bugs of a different flavor that bothers users more. It's optimized for passing tests and convincing people and AIs that security holes are closed, which means other considerations like preserving functionality can more easily be regressed as compared to if humans were doing it. (If true, it still doesn't support the claim that depending on AI code is a catastrophe, fwiw.)

I'm not arguing the conclusion is wrong. I'm saying the analysis proves far less than it claims to. As for whether it's a debacle for rsync to become dependent on AI code generation, I think that's a reasonable debate to have but it's not going to be resolved this reductively.

Wha? Why didn't they use Q=K=V for that?

This was pretty directly addressed in the article: not doing it would only mean they'd fall behind whoever would. This is not peace time in the AI race.

Whether you agree with that argument is another question.

The decision wasn't specifically to drop a standard bar. It was to drop the existing bars because they have become heavily gamed and are far more reliable indicators of your family's resources than your ability or likelihood of success. That was the equity argument.

Unfortunately, the lost signal wasn't replaced with anything. (I don't know what could replace it. It's an incredibly hard problem. )

Is flunking kids the right reaction to catching them cheating? If it was before LLMs, is it still? I would love to be able to hold the line and throw the book at anyone who cheats, but after the dam has burst does it still help to try to hold the water back?

The whole situation sucks for both students and teachers. Teachers know that the knowledge they're going to great effort to convey isn't going anywhere. Or at least, it's landing in far fewer fertile brains than it used to. Students are squeezed because part of the university experience is being forced to adapt to an academic load, and as a result change yourself in ways that benefit you (or at least produce learning!) There have always been relief valves -- not just forms of cheating, but blowing off a study session by using game theory on your grade or going to a tutor or taking easier classes or extending your stay at the school. But now there's this huge giant relief valve in the form of a shiny LLM that is always available, particular at 3:45am when your project -- the one you've steadfastly refused to use AI on thus far -- is due the next day. The schools have tuned the pressure for the old set of options, and it's not clear that there's a new tuning that maintains anywhere near the old level of learning.

I guess my question is: of those students who were flunked for cheating, how many of them were learning despite their cheating? (And how about the students who were cheating but not caught?) Also, what levers are there to move more students towards learning even with the chatbots present?

I'm sure these questions are being debated. I know Garcia personally, and he is very invested in his students learning. The title of his Joy course is legit. So I'm sure the profs have ideas around this, though clearly not happy ones. Perhaps I'll ask him.

I doubt that's the bottleneck. UCB's acceptance rate is not high (<5% for CS). They have way more people who want to get in -- qualified kids, too! -- than they can fit. They'd need to burn through that backlog before it started showing up as a signal.

Good point. I'm inclined to think it's because of a quality threshold in chatbots.

Not if kids are prepping for the test in a way that results in real gains. Which seems likely, especially in the age of AI: "should I actually study math or just use ChatGPT to pass this course?" One semester of coasting through might not do that much harm, but at some point the compounding effects will tip you over the edge.

I disagree with you, for the very reason you give:

Then again, maybe someone will just make a LLM that’s built to turn poor English... into excellent English

That's already been done, for some (pretty weird) definition of "excellent".

I work with, or at least in the vicinity of, someone who is very good at getting work out of LLMs. He has a whole system of CLAUDE.md files and skill files and things. He makes TONS of typos. When I first saw that, I was itching to go in and fix them all, it seemed viscerally wrong to be adding an extra layer of correction required between the instructions and the LLM's behavior. But in practice, I don't think it mattered at all. The LLM didn't care. Typos in particular might require a bunch of RLHF in the chatbot, but my hypothesis is that the LLM is already mapping messy human input to the nearest surface of some high-dimensional manifold and the added noise of typos is inconsequential to where it ends up (as long as there isn't any real ambiguity -- though even there, you could probably construct cases where that would help rather than hurt!)

Typos are different from sloppy writing, but I think the AI companies have put a lot of work into training these chatbots on dealing with typical non-English major writing with all of its imprecision. Also, it's easier to construct cases where that imprecision and sloppiness would help rather than hurt: a mistake in the input that is common enough to show up in the training data is going to be a good match for the needed correction as well as associated corrections. The precise language could easily result in the LLM overestimating the user's competence.

That doesn't address whether an English major's careful composition would help for hard tasks where getting the specification right really matters -- perhaps that was your point? I guess it's an open question whether "boiling away the typos" and "boiling away a poorly articulated specification" are related enough.

Yes, that's fair, and you are right. I guess I was using hyperbole to contrast with the naive and common "I'm great at basketball so obviously I'll be really good at baseball" mentality. The skills are not unrelated, but success in one area does not guarantee success in another.