Slop Cop

by ericHosick • 3 months ago

252 157 3 months ago

Cpoll• 3 months ago

The LLM Prose Tells and wiki page linked in the readme was a fun read. https://github.com/awnist/slop-cop?tab=readme-ov-file#source...

add-sub-mul-div• 3 months ago

I'm sure there are some useful applications of this but we can't trust the reliability of an AI detector for the same reasons we can't trust the reliability of AI.

awnist7• 3 months ago

True. This is just an LLM cliché detector, highlighting stylistic habits they're currently prone to. You'll start noticing them everywhere when you internalize the patterns.

kstrauser3• 3 months ago

But you named it as though it's an AI detector.

rkagerer• 3 months ago

I'm curious how well this thing works, but you need a yardstick to measure it against. The last year or two a burgeoning community of meatspace AI detectors has emerged right here on HN, it might be fun for someone to rank "sloppiness" of submitted HN articles as gauged by comment sentiment vs. this tool to see how well they align.

awnist7• 3 months ago

Good idea, I did test on HN op/ed submissions and likely upgraded my own wetware slop radar in the process

keybored2• 3 months ago

"In an Era of…" Opening phrase that stalls before reaching the actual argument.

Always gotta have In This AI Era of Ours. Because even if you fail to convince the reader of the point you ostensibly were trying to make you still get to tediously skull-bang about The AI Era. And it only costs tokens.

Staccato Burst Three or more consecutive very short sentences at matching cadence.

This is real. It’s not your imagination. AI is here and eating your lunch/AI is psychologically draining/The unemployment lines are unusually long.

korse• 3 months ago

The feedback needs to go away or this thing is just exacerbating the problem. Give a slop score if you must but then shut up and let the user interpret the result as they see fit.

Slop is stopped by allowing unique quirks to flourish. Do you speak in 'staccato bursts'? THEN FUCKING WRITE IN STACCATO BURSTS! Do you need a 'throat clearing opener? THEN FUCKING USE ONE!

Human language does not need to take progressive steps toward some universal standard. Having one is fine, in theory, but the beauty lies in how we solve for our inability to consistently utilize it. Adding mechanism to every step removes the beauty. Stop being the problem.

Chaosvex• 3 months ago

Agreed. I ran some human-authored technical articles through this and most (all?) of the suggestions were just stripping the personality out of the writing. Kind of ironic.

ameliaquining2• 3 months ago

It seems relevant that a lot of these things were fairly notorious clichés even before LLMs, which just intensified the phenomenon. They were what people tended to do who wanted to sound smart and sophisticated but didn't have a developed voice or anything in particular to say. Indeed, I'm fairly sure this is why LLMs sound like this.

cryzinger• 3 months ago

This is true, although I can still get behind "use fewer cliches" regardless :)

keybored2• 3 months ago

AI is so original that it can’t make cliches out of decently-worn phrases and constructions by itself.

gus_massa• 3 months ago

It's just following what the prompt says, something like:

fake prompt> To sound smart, use as much literary tricks from LinkedIn Grow Hackers as possible.

If they prompt asked to sound like Strawberry Shortcake, the AI pudding would be full of berry interesting cooking analogies.

justzisguyuknow• 3 months ago

Indeed,

layer8• 3 months ago

A lot of these things were well-known clichés already before LLMs, used by people who wanted to sound sophisticated but weren't articulate or didn't have anything to say. That's why LLMs sound like that.

riwsky• 3 months ago

Artificial intelligence predates computers.

sublinear• 3 months ago

I don't understand the point of this. Terse writing isn't always necessarily better or something that LLMs are incapable of.

This doesn't detect AI slop. It's just a grammarly/copilot clone.

RogerL• 3 months ago

You want me to enter my api key into a website?

Yes, I see the message about it staying local. No, I don't trust the message or that you will never be hacked.

awnist7• 3 months ago

Run it locally, Github is linked on the bottom left:

https://github.com/awnist/slop-cop

kstrauser3• 3 months ago

LOL. I copied and pasted an 87-word blog post I wrote yesterday, on my phone, via my own thumbs. It detected 4 likely AI patterns, or once every 22 words.

I'm so over this idiocy. It's gotten to the point that the "haha, gotcha!" AI claims are more annoying than AI slop itself. God forbid you use a semicolon or an em dash or an interesting sentence structure to break things up, because someone will be quick to point out the "proof" that it's machine generated.

awnist7• 3 months ago

It isn’t an AI detector. It flags valid language patterns that have become LLM-output clichés through overuse. False positives are a given.

and I'll never give up on em dashes

nz• 3 months ago

I've taken to telling people, that if they see me write a long piece, that lacks em-dashes, then they should assume that I am under duress, and send help.

tavavex3• 3 months ago

This is a confused and misguided project. It makes the mistake of failing to identify why the AI 'style' feels wrong. The author decided to replicate similar tools by breaking down AI writing into bite-sized issues, but it just doesn't work the same way as correcting grammatical errors. Because of this, the author had to really try to find what's so wrong about these patterns in isolation, so all of it comes off as annoying nitpicks. Let's take a look at a few.

Overused Intensifier - Delete it. If the sentence still makes sense, the word was never needed. If it doesn't, rewrite the sentence to show why it matters.

You heard it here first. Adjectives? More like AIdjectives, a covert plan by AI companies to make our writing more sloppy. According to this recommendation, writing should never have any emphasis, it should only contain the most basic "X is Y" relations, like in some programming language. Sentences should contain the bare minimum amount of information required to parse them, everything else must be cut. In practice, this recommendation only filters a few of the most pervasive 'corporate PowerPoint'-style language, but even then, the suggestion that these words are never useful is wrong.

Triple Construction - Break the pattern. Use two items or four. Or convert one item into its own sentence to give it more weight.

Humans may really like when things are structured into threes, but you must resist this AI temptation! Use two or four points, because you're not like them. The only reason cited for why this is wrong is that LLMs use this pattern often, so naturally the rest of us must cede good writing practices to them.

"Almost" Hedge - Commit. "Almost always" → "usually." Or just say "always" and defend the claim. Readers notice when you won't take a stance.

As we all know, the world is discrete and easy to describe. That's why there simply isn't anything between things that happen "usually" (70%) and "always" (100%). Saying "almost always" (95%) is bad, because you should round your estimates and defend what is now an obviously wrong statement, for it makes you seem more brutal and confident.

"Broader Implications" - State the implication explicitly, or cut the phrase. "This has broader implications" says nothing. What are the implications? Say them.

God forbid you organize an essay that's in any way non-linear, temporarily withholding some information for the sake of organization. Asking to can the phrase entirely says that even complex writing should be strung together in a rigid and sequential order.

That's the problem with the project, the way I see it. It was too heavily inspired by Grammarly and the likes, and in chasing it, the criticisms were adapted to fit the Grammarly model. The issue with that LLM 'style' is the punchy, continuous overuse of these patterns to the point where these phrases start seeming like meaningless sound combinations. There's nothing wrong with most of these patterns individually, what I hate is when text is filled with them to the brim, not when they show at all. If your writing is like the example paragraph, with most of the text highlighted, then it's a sign that your essay is more rhetoric than substance. But if you write an argument with three items in it and it's highlighted because "that's like AI" to make you delete it, then that's performative self-censorship, not improving your writing.

aesthesia• 3 months ago

I think this would come off a lot better if the recommendations weren't so absolute. I like the effect of a multicolored slab of highlights calling out every LLM cliche in a passage. Yes, the slop style is not just the sum of these individual patterns, but they're definitely significant contributors to the effect, and they're worth being aware of in your own writing regardless of their association with LLMs. You just can't treat it as a list of must-resolve errors (same as with any writing feedback, really).

lemagedurage2• 3 months ago

Yeah, "don't overuse these patterns" is the right attitude for tools like this, not "fix all mistakes". And that's OK?

tavavex3• 3 months ago

It would be OK, but the point I'm raising is that the Grammarly-like design encourages the user to resolve everything it highlights, to make the text look uniform and spotless.

dingaling• 3 months ago

According to this recommendation, writing should never have any emphasis...

If you have measurable amplifications, use them. "This outcome was 40% more frequent". Otherwise keep subjective emotion out of documents, unless you're writing a novel.

God forbid you organize an essay that's in any way non-linear...

Essays should be brutally logical and sequential. If the text is becoming cluttered with data, break it out into a table. I read a document for information, not for some movie-director suspenseful build-up and revelation.

There's a good rule where I work that any document that requires someone to make a decision must fit on two or fewer pages. Anything longer is TLDR. Tables and charts are prized for their information density, novelesque writing is not.

tavavex3• 3 months ago

Otherwise keep subjective emotion out of documents, unless you're writing a novel.

There's more types of writing between the extremes of research papers and novels. Data is useful and all, but asking it to be the sole driving component of ALL types of non-fictional writing is too much. Besides, this tool would criticize your novel just the same, because the intended use is to have it filter everything you write.

starkparker• 3 months ago

This is a genuinely useful tool with a shitty self-sabotaging name.

jonahx• 3 months ago

Name seems fine. Catchy, and I knew what I did before opening link.

alexjurkiewicz• 3 months ago

And it's unnecessarily rude. Grammarly and Hemingway can identify the same sort of issues without "you are a stupid robot" vibes.

0gs• 3 months ago

... the problem is it is rude to LLMs? i think they can handle it.

setnone• 3 months ago

'Add Slop API Key' button is a missed opportunity here

IshKebab4• 3 months ago

What do you mean? It's a great name. Catchy, rhymes, and immediately tells you what it does.

chromacity2• 3 months ago

Like the original Grammarly, I think this can be useful for business writing because these tools help you get to the point. Many students are rewarded for using flowery language in school essays, but if you're composing an email or writing a design doc, just optimize for reading time and clarity.

But for general use, I think this is misguided. The problem with LLM output is not that it's using em dashes or words such as "crucial". It's that most LLM articles on LinkedIn or on personal blogs just take a one-sentence prompt and dress it up into a lot of pointless words, wasting everyone's time: "I had a shower thought and I asked a chatbot to write five pages of text about it." I don't need prettier words, I need there to be far fewer of them?

On the flip side, if you're a human and actually have something of consequence to say, "delve" all you want.

Swizec• 3 months ago

"I had a shower thought and I asked a chatbot to write five pages of text about it." I don't need prettier words, I need there to be fewer of them?

Always judge an author by the length of their text.

Decades of insights barely condensed into 200 pages? Great! Hours of thought expanded into 200 pages? Very bad.

Same length of text but lands very differently. Same is true for emails, tweets, videos, and even just talking. Say less! But not too little either.

grahamplace• 3 months ago

As the saying goes: “If I had more time, I would’ve written a shorter letter”

littlexsparkee• 3 months ago

So many books that could've been an article. I try to save myself time by checking Goodreads but it's not always clear as I'm more critical than the average person. Reading a preview in Google Books helps but you only get so many pages before you're cut off. Appreciate that lately new books are sometimes featured in pubs with an excerpt.

card_zero• 3 months ago

So, not in fact the length of the text, which is constant at 200 pages.

ChrisMarshallNY• 3 months ago

So it’s really about the content; not the metrics.

My mother was British. She was also an awesome cook.

She used to say that the British dining table was the fanciest in the world, with fine china plates, silver silverware, lace tablecloths and matching napkins, etc., but terrible food.

French tables, on the other hand, were casual affairs, with newspaper on the table, and a candle jammed into a wine bottle, but excellent food.

Aurornis• 3 months ago

Always judge an author by the length of their text.

Flashbacks to a past employer where the CEO decided that brevity was a core company value and started rewarding people for short communications and scolding us for longer text.

Over the next year a few charlatans moved up the ranks by spitting out half-baked thoughts and e-mails all the time, which looked like clarity and brevity on the surface. People were afraid to speak out or discuss nuance because it was too many words, and you didn't want to use too many words.

eduction• 3 months ago

What you say makes no sense by your own logic. 200 words can be wonderfully filled with wisdom or devoid of insight depending how much work and experience went into those words. So it is not appropriate to judge an author by the length of their text. You need length/wisdom but you can’t objectively or quickly determine the denominator.

tombert• 3 months ago

Tangential, but I remember when I was studying for the ACT, there was something in one of the practice books that stuck with me. I'm paraphrasing but it was something like "Good writing is clear and easy to understand. It's about communication, make sure you communicate".

It was something that I guess I logically knew but hadn't fully realized. I had always tried to be fancy with my writing and pad it out to meet minimum word counts, with "understand-ability" being somewhat of an afterthought. Just that one statement in my ACT prep book made me, in my opinion, a significantly better writer.

teddyh• 3 months ago

I recommend <https://www.youtube.com/watch?v=vtIzMaLkCaM>, Larry McEnerney’s lecture The Craft of Writing Effectively.

golem14• 3 months ago

Check out the books by Rudolf Flesch. Old school, but ever more applicable. Also, Bugs in writing, by Lyn Dupree.

Angostura• 3 months ago

Good writing and good communication is also about keeping the reader engaged and concentrating, however. Even in business writing - for example, how-tos or intranet pages, altering sentence length, using rhetorical questions can be helpful. I'm concerned that tools like this will tend to stamp out useful writing conventions, that were picked up by LLMs precisely because they were useful.

The result? Increasingly homogeneous, boring text.

moron4hire2• 3 months ago

This is something I've been working on in my own professional writing for years. I used to write very long emails, thinking I was providing insight and detail, but nobody would even bother reading them because it was such a chore.

I consider more than three paragraphs and more than two sentences per paragraph a "writing smell." It's relatively easy now, especially when I realized my predilection for verbosity was actually a symptom of my own insecurity, emotionalism, and indecisiveness.

I try to limit my emails to one, clear, strong point—usually just factual statements—in the active voice, eschewing adverbs as much as possible. The emails almost write themselves now, because there isn't much choice on what to write anymore.

edaemon• 3 months ago

Yeah, this comic summarizes the issue pretty well: https://marketoonist.com/2023/03/ai-written-ai-read.html

lokar2• 3 months ago

As a senior engineer I spend a lot of time reviewing and approving technical designs, PRDs etc.

Over the years the amount of basic copy editing I have to do has really grown. I sometimes feel like I’m removing 20%+ of the text. And that was before LLMs.

abustamam5• 3 months ago

For better or for worse my team has standardized on using Miro for technical designs and diagrams. It's a lot easier to visualize the system in a diagram than it is to talk about it in prose.

I think it's important to choose the right medium for communication though. Some things just need to be written out concisely.

tedggh• 3 months ago

Ask your team to make their documentation at least 30% shorter just before sending. I don’t remember where I first I read this advice for writing code, but it’s been part of my workflow for several years. It’s an arbitrary number, but it forces you think how to make things simpler. I apply the same principle when packing for travel or hiking.

slopinthebag• 3 months ago

Many students are rewarded for using flowery language in school essays

Not to nitpick, but I actually had the opposite experience in uni. My prof docked me marks for my flowery language, and honestly, good for her, my lazy writing style honestly sucks (see how I used "honestly" twice in the same sentence, lol).

Not to take away from your post or anything, just realising I got lucky with my prof. I agree that LLMs produce way too much output when generating writing (and code too!)

flexagoon2• 3 months ago

In uni, maybe. But my experience in middle/high school was that hitting the minimum word count was much more important than actually good writing.

jollyllama4• 3 months ago

The problem is not X. It's Y.

Your writing style, if not your thoughts, have already been infected by LLM prose.

sdthjbvuiiijbb• 3 months ago

Nonsense. It's a common construction that LLMs didn't exactly invent. I don't think their usage evokes LLM writing at all (not short and punchy enough).

chromacity2• 3 months ago

No. I've been using that construct long before LLMs and I don't think there's anything wrong with it. It allows you to succinctly state the position you're disagreeing with before putting forward another hypothesis. LLMs overuse it for needless emphasis, with the negative example usually reduced to a single word.

comex• 3 months ago

Just because LLMs overuse it doesn't mean it doesn't have its place.

The way the OP used the 'not X, but Y' pattern, the 'X' and 'Y' are two clear, specific, and (most importantly) distinct things, as opposed to stereotypical LLM usage where they're vague characterizations or metaphors. And there's a reason to emphasize that it's not X, because the Slop Cop website implicitly suggests that it is X.

tom_• 3 months ago

This doesn't apply here - I don't think? The article claims X; so it is surely no sin for the post rebutting it to straight up state that X is, in fact, not the case.

The LLM tic, by contrast, has a noticeable tendency to be deployed even when X has never been previously mentioned. It is a valid rhetorical technique, and I assume that's why the LLMs have picked up on it - but it has to be deployed judiciously. Which is something LLMs appear absolutely incapable of doing. And that is why people notice it, and think it sucks.

UqWBcuFx6NV4r3• 3 months ago

Anyone over the age of 25 actually developed their writing style before ChatGPT came about. Getting all uppity about these surface-level LLM ‘tropes’ is just stupid. I am thankfully yet to run into a situation where someone with this attitude is actually in a position to be able to negatively affect my life. I’m sure that there’s a correlation. Take the “ew, em-dash” stuff back to Twitter.

Teever• 3 months ago

Flowery language is important but something like BLUF - Bottom Line Upfront[0] is important too.

While it's important for universities to continue to teach the ability to write using 'flowery' language I think that it is also important that schools teach students something like BLUF -- Bottom Line Upfront.[0]

Compare and contrast those two sentences. I'm fine writing a comment that us just the first sentence and the link without a footnote but I know as a message it won't go over well on a site like Hackernews. They looooooove their verbosity here.

So in some situations you have to gussy it up -- give it some of that Emeril "BAM". The deal is that you have to know your audience. The medium is the message.[1] shit like that.

Stuff on Linkedin is full of pointless words because that's what Linkedin is for -- it's about signalling to other people that you can string together a bunch of pointless words that are effusive and vaguely passive aggressive at the same time -- you know, typical business shit.

“Whether in a suit or in a loincloth people are ignorant little thorns cutting into one another. They seem incapable of advancing beyond the violent tendencies which at one time were necessary for survival.”

We can delve into this kinda stuff but really it just comes back to the know your audience and that the medium is the message. Also don't repeat your self.

Definitely don't repeat yourself.

[0] https://en.wikipedia.org/wiki/BLUF_(communication)

[1] https://en.wikipedia.org/wiki/The_medium_is_the_message

danilocesar• 3 months ago

I've heard this theory in the past:

In a couple of years, the corporative communication will work like this:

You write a bunch of bullet points and feed them to an AI to create a beautiful and well written email. Your reader will feed that email into his own AI and he will generate bullet points to read.

gattr• 3 months ago

In other words, just as they did in "Rob Roy": "My factor will contact Your Grace's factor..."

red_admiral3• 3 months ago

I've heard the story of the time e-mail was new and one secretary's job was to print out her boss's emails, he'd write a reply below by hand, and she'd type it back in and send it.

cineticdaffodil2• 3 months ago

"Can you condense that down further without loosing clarity? " goes into every prompt.

6031769• 3 months ago

I can condense that and improve clarity simply by s/oo/o/

rdmuser• 3 months ago

I've been wanting something like this for a while now but as an extension that runs locally. Just something I can click to get a quick response telling me if the article seems like ai so I can focus on the writing without needing to spend energy on remembering ai styles and detecting obvious ai. I'd be pretty happy to see something like that built straight into firefox.

Ultimately slop is so pervasive that I'm wasting a fair amount of time vetting text and it's affecting my ability to simply enjoy reading. I keep getting part way into an article before realizing it's low quality ai writing. Being able to get a quick heads up that it looks like ai before starting would save me a lot of energy even on articles I decide to try reading because it cuts down on mental overhead.

doginasuit• 3 months ago

It sounds like you might be overthinking it. Slop is pretty noticeable at a glance, and it doesn't really matter if it is AI slop or human slop. On the other end, I have probably enjoyed an article or two that was made partially or entirely by AI and I'm not sure what the downside is.

furyofantares2• 3 months ago

Removing the aesthetic tells from LLM-generated text won't fix the fact that there's nobody home with opinions and experiences to express. It will just make it take more work for your unsuspecting audience to figure that out.

IshKebab4• 3 months ago

Yeah but there's a reasonable amount of content that has real information from a real human but they've used AI to help write it. Maybe they got a first draft from AI and then fixed it.

Those can still be things that I want to read but the AI rhetorical style is so tedious and overused at this point it's really annoying to read them. So this tool would help with those cases. (Assuming people actually use it.)

furyofantares2• 3 months ago

I believe you, but I have yet to see any I want to read.

I've had 5 or 6 times where I've thought, OK, finally someone has produced something useful this way. And in the end I've always been bitten, by hallucinations or by an inability to work out what the author cared about or was trying to express.

Again, I believe you've had ones where you did want to read it, I'm not trying to contradict that. I'm still waiting though, and until then I'd really prefer the aesthetic tells to still be obvious.

iamjs• 3 months ago

I'm enjoying pasting early 2000's era blog posts in here and learning that they too were LLM slop!

lotyrin• 3 months ago

It's not purporting to be an LLM detector. Your 2000s era posts probably do have some sloppy cliches in them.

rorylawless• 3 months ago

And it is not impossible the posts ended up in the training corpus.

Mistletoe• 3 months ago

Staccato Burst.

Now I have a name for the thing I despise the most about AI writing.

krackers• 3 months ago

Isn't this called the tricolon? Ironically the names of the patterns all seem AI generated.

lemagedurage2• 3 months ago

It looks like tricolon is about specifically three parallel elements, while staccato is about short consecutive sentences, so staccato would be the appropriate name here.

cjlm• 3 months ago

I find it funny that all of these little tools lean into the slop = poop dynamic.

I'm building writetrack.dev - a writing signal sdk that helps folks understand proof of process. It takes a different approach to writing analysis and I'm pretty sure the logo will never feature a brown turd.

3eb7988a1663• 3 months ago

Wow! Did you know that Abraham Lincoln let AI write the Gettysburg Address? 17 patterns identified out of only 305 words. I don't know why I ever let him get the ad revenue.

NitpickLawyer2• 3 months ago

Haha! Did the same with Asimov. These "detectors" are a joke.

IshKebab4• 3 months ago

I tried that but most of those seem to be false positives. It seems to report Triple Construction based just on commas, which were apparently way more common in writing of the time. It also reports em-dashes which are obviously irrelevant in this case.

And nobody is really saying that you need to completely eliminate these constructs. 17 matches out of 305 words is an order of magnitude less than the example it opens with.

resters• 3 months ago

If prompted appropriately, LLMs can write prose that mirrors nearly any style you ask for. Not sure what the big hubbub about "slop" is. Honestly it's annoying.

m0llusk2• 3 months ago

That isn't really true, though, at least not when actual common usage is considered. As the studies of "Trendslop" have revealed, LLMs aggressively normalize output. What they do is generate content based on averages over large samples. This gives everything from them a strong tendency to revert to the mean from concepts to presentation and style.

burnished• 3 months ago

I love how you can tell something was generated because it comes out clear and using many of the important writing fundamentals we were all taught.

m0llusk2• 3 months ago

Being cautious and an autistic mathematician also I am prone to heavy qualification. This causes very large blocks of my writing to be highlighted as "Hedge Stack" which isn't really helpful. Lots of Overused Intensifier and Triple Construction instances also, but those are usually words or phrases, not several paragraphs together as with Hedge Stack.

Seems like a sad situation, but I'm not going to start changing my communication style to avoid sounding like an LLM. At least not yet.

foxfired• 3 months ago

One thing I learned is that AI written text is not hard to spot. Usually, when I meet slop, I close it one or two paragraphs in. Although tools like this will become more common, they usually serve to win an argument, or confirm what you already believe.

Also, it was painful to learn that my very first blog post I wrote in 2013 is AI generated. But I'm fine with it because I read this:

A short punchy opener (≤10 words) followed by two or more substantially longer elaboration sentences — the LLM "hook then evidence pile" rhythm.

... and realized that the entire app is AI generated.

jillesvangurp• 3 months ago

If you can spot it, an AI can spot it too. We have a website with some AI generated content (about AI). I added a skill to correct AI slop. Content got a lot better when I put that in place. I actually made codex research slop patterns and it came up with a list of known AI slop linguistic anti patterns. It now fixes its own content using that list. I also put a guard rail in place to do a critical review of all produced content as a final quality gate. That actually catches a lot of baseless claims, and other slop. And there's another skill that ensures we use the right SEO relevant language (a list that is produced by a separate agent).

It's actually starting to generate interesting content based on me giving it a few bullets and ideas. I won't claim it's perfect but it does a decent enough job.

I have my reasons for doing this (we help people set up agentic work flows) and I appreciate that not everybody likes the idea of AI generated content. But I think it will start getting harder and harder to spot AI slop. Basically slop is what you get without guard rails and quality gates. Of course, most people still lack the skills to configure their AI tools properly. Particularly non technical people. But it's not that hard and I bet there are a few handy journalists out there getting better at this. Also, for technical writers this is not going to be optional.

koito172• 3 months ago

How does this site tokenize text? Split on ASCII whitespace?

Inputting Japanese sentences of any length flags the whole sentence as "Dramatic Fragment: A standalone paragraph with ≤4 words".

awnist7• 3 months ago

That's right, its tokenization and fragment rules use fairly simple heuristics that assume whitespace delimited words plus English language/punctuation. Proper CJK support would require language specific tokenization and morphological parsing. Correcting rules like "≤4 words = dramatic fragment" would be difficult. The more complex rules already require LLM roundtrips, so supporting all languages in one pass would need to rely on LLM alone, I imagine.

Which brings up an interesting point - what do these LLM clichés look like in Japanese?

https://github.com/awnist/slop-cop#client-side-instant

koito172• 3 months ago

what do these LLM clichés look like in Japanese?

Besides text reading like a machine translation, the tell-tale signs often involve things like:

- itemized lists (I know, it's ironic that I'm using them here)

- frequent use of conjunctions

- use of demonstratives that feels redundant

- full-width colons, especially in titles

- subheadings that always end in abstract nouns

- bold text, especially at the beginning of a line

The demonstrative bit may be hard to express, but to give you an idea: when communicating in Japanese, words that can be understood from context may be omitted. Explicitly writing out words understood from context can sometimes make a sentence sound redundant.

Before LLMs were widespread, SEO spam in the Japanese net tended to be affiliate sites with predictable, template paragraphs. I get reminded of those sites whenever GPT starts a response with 「結論から言うと、〇〇」, since that's exactly how those affiliate sites wrote back in the day.

doctorpangloss• 3 months ago

gptzero.me is an excellent LLM-authored text detector.

trane_project• 3 months ago

I pasted a blog that I wrote myself and flagged hundreds of patterns. Granted, the article is 15,000 words so some are expected but there are simply too many false positives to make this and any similar tool have any usefulness beyond flagging the most obvious offenders.

And looking at its suggestions, they are not very good. People are better developing their own writing style than trusting generic advice meant for common-denominator writing.

JuniperMesos2• 3 months ago

I did the same thing, and also found the results pretty mediocre. At best it did point out some places I could've dropped a less important word to be more concise, but pretty much any editing will do that (and I do think I tend towards verbosity as a writer; it's a flaw, but it's my organic flaw). Beyond that it was just pattern-matching on sentence structures it thinks are over-used in LLM-generated prose. Even if that's true, it's not a good argument for why I should rewrite a particular sentence in context.

trueno• 3 months ago

yup same just tried it myself which an email I wrote.

like llms are using our language here so there's gonna be patterns we use that they also use.

i dont think we're ever going to get to a place where we have a "yes/no this was written by AI" kind of tool. i do be worrying what this might do to the fabric of society but there's no puttin this damn cat back in the bag so i guess we'll find out

NitpickLawyer2• 3 months ago

I c/p a section of Asimov's "The Last Question", since it was readily on the front page. It detected 14 patterns (2 reds, one yellow, bunch of green and blue) in 583 words. Welp, I guess it's back to school for mr. Asimov...

Update: 13 patterns in 800 words for Samuel Clemens. Apparently he's an em-dash abuser, but also likes "filler adverbs", "triple constructions" and "anaphora abuse". Damn!

And for Mr. Hemingway we have 43 patterns in 1600 words. 16 filler adverbs, 5 triple constructions, 5 staccato bursts, and 14 question then answer. My my...

thih92• 3 months ago

What’s the use case?

Paste AI generated text and get a more human sounding version? That’s just AI generated text with extra steps.

cientifico4• 3 months ago

Unpopular opinion.

Until now, ideas were only relevant when the owner was able to communicate then regardless of the impact of the idea.

LLM "democratize"(VC term) sharing ideas, as people with low communication skills can be heard.

thih92• 3 months ago

Could you give an example?

cientifico4• 3 months ago

My self.

LLM helps me communicate my ideas better.

Thinking in different angles, focus on the main idea, structure in a post series... It constantly challenge my mess.

Opus and I, iterate over 20 times a single blog post.

gbalduzzi• 3 months ago

I created a gem on Gemini (the equivalent of a gtp) basically with a set of instructions for rewriting my text in a professional, clear and concise way and it works great.

I just write my text without too much thought about it and I get a rewritten version that is usually clearer, but not pedantic or overly verbose.

It particular helps for English text as it is not my first language

efilife• 3 months ago

This website itself looks like it was generated by an LLM and its github readme as well. Loses all credibility

zombot• 3 months ago

A handy tool. Bookmarked.

And good to know that Teddy Roosevelt was not an LLM: https://www.trcp.org/2011/01/18/it-is-not-the-critic-who-cou...

cientifico4• 3 months ago

I find some parallelism between writing articles and Pull requests.

We are moving to a point in time, where we don't care if the PR was written by AI. We care that the author understand what is about, that it tested it and in general, we want the ownership.

With articles is the same. I don't care if it was written by AI, if the content is interesting, and ai make it easier to digest... That's a win win.

The problem is not the presentation. Is the content.

pocksuppet• 3 months ago

If the prompt is interesting, why not just post the prompt?

cientifico4• 3 months ago

In my case. The prompt is normally a collection of ideas connected over time. Ai groups, structure, challenges and help me organize that ideas. Then, once I see something that I consider worth sharing, I ask to draft a blog post. 20 iterations over, and I have a blog post.

The prompt is normally larger than the content generated.

shevy-java• 3 months ago

Companies that utilize these tools will thrive

I read this before but I have some doubts. I recall some companies that were surprised when suddenly the prices were increased. Usual examples include Amazon, Google and some more, but this can happen to any company, including AI slop master companies. I am not at all claiming that the AI slop has zero use cases, of course - there are use cases, so I don't deny that. But the assumption generated here by AI slop, claiming how all the problems will soon have been solved, and risk-free profits are to be made by all companies, is just rubbish nonsense. AI slop is a big liar. In fact: I am beginning to believe that the current US administration is an AI slop brigade. Every time the stock market yields some suspicious profits, it seems to be that the AI slop protects some thieves here.

zjp• 3 months ago

This is the Hemingway Editor by another name, and I have the same complaint about it that I have about the Hemingway Editor. See:

https://zjpea.substack.com/p/an-llm-will-never-say-thou

mrob• 3 months ago

Great article. I see it was already submitted once and received zero comments, but it deserves more attention than that.

red_admiral3• 3 months ago

The rule of three was actually good and common writing advice in the pre-LLM era. There's psychological studies that show 3 is a good number to grab human attention, which is why you have "an Englishman, an Irishman, and a Scotsman walk into a bar" jokes. Even my Algebra professor said that a well-written definition should have three conditions, for example a group is a set with an operation that is (1) associative, (2) has a neutral, and (3) has inverses. I don't think he was completely joking.

(For stories with multiple protagonists, the common choices that seem to work best for readers are 3 or 5. Humans are weird.)

I suspect that LLMs use that rule so much, because it's so common in their training data, for good reasons.

JuniperMesos2• 3 months ago

A group is an algebraic structure, and many algebraic structures are defined as sets with an operation that obeys some number of axioms. It happens that a group is a structure whose operation has exactly three axioms; but a monoid for instance is a set with an operation with just two axioms - it's a group without the "has inverses" axiom - and I don't think this makes the monoid worse than a group in any meaningful way.

nxpnsv• 3 months ago

In "fixing" the last sentence in the example I got "One can say that we are at a moment in history." Indeed.

cadamsdotcom2• 3 months ago

Nice work. But it only goes half way.

It should loop the LLM’s results back on itself repeatedly, behind the scenes, until its writing is free of signs of slop. After your quality gates pass and the result is presented, it’d be cool to then see a visualization of each of the agent’s drafts that the user can page through to watch how the writing was gradually incrementally improved by the model!

No need to keep a human in the writing-improvement loop. Just show it when it’s slop free.

awnist7• 3 months ago

There is a one shot rewrite function now, but you're right, even when asking the LLM avoid some of the patterns, it will stubbornly repeat them. It's a bit more reliable with smaller fragments of text.

cadamsdotcom2• 3 months ago

I am saying, keep reflecting its attempts back on itself. Over and over again, dozens of times if needed. We’ve seen it - any aligned model wants only to achieve its goal. But it does need to see all of its past attempts and where and why each attempt got a failing grade. That’s just a standard conversation history.

It might spit back the same thing the first round. But after the first time it received the exact same feedback for saying the same thing, the model will realize it’s in a deterministic sandbox and try something different. You need to give it all of the conversation including its past attempts as context. If it tries the exact same wording that’s okay, it’s just one more invisible round of back-and-forth. The model is going to rediscover how to work with the harness every time, but that’s not your users’ problem because you’ve hidden that wrinkly bit behind the automation - they just see “model did 10 drafts and here’s the result - would you like to view the result or page through the drafts?”

What I am describing is exactly what a human would do, it is just automated and thus, getting to a good result becomes insanely faster.

Epholys• 3 months ago

When I copy/paste some text I wrote online, even short, it's full of alerts. Maybe I write exactly like a LLM in English? It's not my native tongue, maybe that's why.

Otherwise, almost nothing. I don't know if it's because it's specialized on English or if learning it as a second language makes it really unnatural?

onchainintel• 3 months ago

Could you still utilize this tool running a local model (any Ollama model for example) instead of making calls out to Anthropic to avoid costs while running this tool or does it have something specific to Anthropic that requires it?

awnist7• 3 months ago

Good idea! Nothing special about the Anthropic calls, would be simple to point at local models

owenthejumper• 3 months ago

My biggest slop peeve is not necessarily using words like 'delve', it's the algorithmic structuring of headings.

Every single article out there is now structured as: - THE problem - THE solution - THE proof - Why it matters

seqizz• 3 months ago

A whole new generation is coming who learns how to communicate by llm-generated text, so I assume by time these used "catching" methods will be invalidated.

zaptheimpaler• 3 months ago

This is so dumb. LLMs have some patterns they copied from people, so now people need to stop using those patterns? Not to mention 10000000 famous books, stories, speeches, essays, comments and more are flagged by these stupid tools. These kinds of tools only narrow the scope of our expression and over time can suffocate the entire written language into a single flavour of corporate approved not-slop speak. Yes AI writing is bad, but sometimes the cure is worse than the disease.

squibonpig• 3 months ago

Corporate speak sounds like AI and vise versa though I don't think this concern makes sense.

xnx• 3 months ago

Slop Cop

Thought this was a NY Post-style headline about FBI "top cop" Kash Patel's drinking problem: https://www.theatlantic.com/politics/2026/04/kash-patel-fbi-...

Noaidi• 3 months ago

Oh. My. God.

I had a suspicion that a friend was using AI to respond to my texts///and this said he was!!!

It caught a "Short-Hook Paragraph" and "Negation Pivot" and "Staccato Burst" in one text.

Wonderful tool!

kmfrk• 3 months ago

Looking forward to putting every apology statement into this - although many of them tend to require OCR for being posted as images or screenshots.

Slop Cop

Related Stories