The Information Theory Behind Why AI Writing Sucks

https://www.pangram.com/blog/joe-stech-information-theory-why-ai-writing-sucks

by mojoe • 2 months ago

3 1 2 months ago

davnicwil• 2 months ago

interesting thing about the scoring effect narrowing the types of writing that generate the higher average ratings.

I guess also there's something fundamental about next token prediction that is going to narrow in on 'stable' ratings. Let's say for the 4/5 rating you have a wide range of styles, there's also this question of then the style of 4/5 isn't going to be cohesive, so which one do you pick?

Any output that mixes them up is going to get rated lower in later testing, and then if you have somehow to pick one, you've got to stick with it, which seems like it would be complicated to put into next word prediction weights. You'd need a different branch of weights for each style or something.

So not only does the rating process itself push towards a generic style, but the model 'prefers' training on this generic style, i.e. gets to better-seeming results faster.

The Information Theory Behind Why AI Writing Sucks

Related Stories