LLM-assisted writing in biomedical publications through excess vocabulary

https://www.science.org/doi/10.1126/sciadv.adt3813

by em3rgent0rdr • 1 year ago

108 107 1 year ago

"Delves", "potential", and "significant" all made a large increase. Those words also strike me as the non-committal weasel words Google's AI overview for Google searches tends to produce as the AI isn't really entirely sure it can make a definitive conclusion.

tiahura2• 1 year ago

Potential I’ll grant you, but how are significant and delves weasel words?

dmead• 1 year ago

If you're reading a paper you're already delving. It's clearly filler.

leakycap24• 1 year ago

What does "significant" mean? It means whatever I say it means, it's fairly useless as anything to convey a fact without a definition.

If I needed to program a light sensor to signal when it detects "significant" change in lighting, what does that mean? If I don't define it, it means nothing.

"Delve" is a style word, another category highlighted in the article as a signature of generated text.

leakycap24• 1 year ago

Fascinating to see that the word "important" dropped off in usage

Given the way generated text works, it makes sense it would less often highlight a directly important phrase or passage or detail, since it has no understanding of what it is writing.

userbinator2• 1 year ago

"delves" is stereotypical LLM-ese to me. I don't think I've ever heard a real human use that word before LLMs became common.

sincerely• 1 year ago

Magic the Gathering players are intimately familiar with the term :)

ummonk• 1 year ago

I’ve only ever seen it used in magic the gathering

ToValueFunfetti• 1 year ago

The dwarves of Moria are notable for having delved to excess.

SoftTalker• 1 year ago

I'd see it occasionally, usually in bullshit corporate writing replete with other buzzwords, which is probably where the LLMs picked up on it.

jowea• 1 year ago

It seems it is common in Nigerian English, from where in ended up in AI datasets.

OtherShrezzing• 1 year ago

I think it’s from YouTube. “With that, we can delve into [topic]” was part of the vocabulary for video essays for years. It always joined an introduction’s closing thesis statement with the rest of the video content.

bongodongobob• 1 year ago

Delves has a huge increase because it's a new thing in WoW. That's why you see a big spike in Google trends. If you see an article that claims "delves" increase is because of LLMs, you can safely ignore anything else it has to say.

derbOac• 1 year ago

Took me too long to catch the playful (or hinting?) word use in the title.

leakycap24• 1 year ago

I had downloaded the article and began taking notes and hadn't noticed until your comment. Oh dear.

pcrh5• 1 year ago

The post title isn't grammatically correct. It should be "caused a" or "shifts"

HappMacDonald2• 1 year ago

This strikes me as the kind of language fingerprinting someone might use to look for things that I write: I tend to discard articles (a, the, things like that) when I'm in a hurry.

accrual3• 1 year ago

I'm not a scientist or someone producing technical literature for others to consume, but I feel like I'd want my humanity to come through on important topics like this. I love using LLMs but there are some tasks (like writing comments or texting friends) that I refuse to use them with, otherwise I'm applying `normalizeText(myUniqueText)`.

I'm sure there are deadlines and other constraints that cause professionals to reach for LLMs when producing their work, just like school-age kids use them for homework so they have more time to do whatever they'd rather be doing, so I can understand.

IncreasePosts• 1 year ago

Ok, but what if your humanity expresses itself best in a language that only 20M people speak with no major biomedical publications?

leakycap24• 1 year ago

your humanity expresses itself best

I don't think these papers are about human expression; they're about communicating data and learned information in a way that can be used by others.

You might be thinking of the field of art, which also has strong opinions about AI in their field.

JuettnerDistrib• 1 year ago

I'm sure many researchers have English as a second language, and rely on LLM to fix their grammar and vocabulary.

leakycap24• 1 year ago

Which is horrifying: if an ESL author must publish in English and therefore does not have a full grasp of nuances and meaning conveyed by English wording, they should involve an editor... not a word machine that doesn't understand, either.

Wording matters when conveying information, ESL speakers should be working with fellow humans when publishing in a language they do not feel comfortable writing on their own.

perching_aix12• 1 year ago

I'm not (...) producing technical literature for others to consume, but I feel like I'd want my humanity to come through on important topics like this.

This will absolutely be just an opinion, but the kind of documentations I dislike the most are the ones that are (full of) arbitrarily structured prose. There's a time and place for self-expression and phrasing liberties; intricate, brittle, and especially long technical descriptions I don't think is one of them.

accrual3• 1 year ago

Definitely, I think I wrote imprecisely when I used "humanity". I meant more like I personally wrote it, the sentence structure and grammar are mine, the mistakes are mine, and hopefully it's still clear and easy to understand.

Speaking of mixing technical text and style/prose, I feel this link from yesterday did a great job executing both (granted it is an article, not a paper):

https://jrsinclair.com/articles/2025/whats-the-difference-be...

mtkd• 1 year ago

I expect in a few days there will be a new tool launched that returns word frequency/velocity in recent biomedical papers ... so next year's PhDs can level things using an MCP function

lawlessone• 1 year ago

Will there? is someone working on it?

A_D_E_P_T2• 1 year ago

Something you've gotta understand is that the majority of English-language scientific journal articles are written by authors who aren't native English speakers/writers.

https://ncses.nsf.gov/pubs/nsb202333/publication-output-by-r...

In the past, just after they submit a poorly-written paper, the sleazeballs at Wiley/Elsevier/Springer would "encourage" said authors to employ their "Author Services" who would edit and rewrite their paper for them, and resubmit it for publication. This didn't come cheap.

Today, everybody just uses LLMs. LLMs are masterful translators of language and concepts. It's a win-win for everybody but Author Services.

Admittedly, word choice and sentence structure are much more limited. (This is not necessarily a bad thing. LLM-written scientific papers can be clearer, and are often more tidy, than human-written scientific papers.) I don't like seeing LLM-written text in journalism, literature, and other media, but I don't mind it so much in scientific literature -- if there's no fraud involved, that is.

leakycap24• 1 year ago

It's a win-win for everybody but Author Services.

Having work that can be edited by a human who understands the concepts –or at least the words— shows a great depth of usefulness to the work itself.

Running your rough draft in one language through an LLM so it appears, at surface glance, to be similar to a publication that a team of humans had been involved in doesn't actually provide value to the author or the reader. Now the author has a paper they cannot say they wrote (or indeed, say who wrote it) and the reader cannot assume that what they read was intended to mean what it now means in English.

If anything, people who write papers in another language to be published in English should be leaning more than ever on human editors and quality assurance measures, not slop machines that hallucinate with good grammar.

DontBreakAlex• 1 year ago

I find it quite insulting that you seem to think that non-native english speakers are incapable of reading the outputs of LLMs to asses if it still means what they intended to say.

A_D_E_P_T2• 1 year ago

I think you overestimate the extent of Author Services' involvement. It's near nil; there's no back-and-forth; you just send them your article and they send back a version with fewer grammatical/textual errors and correct formatting for the journal. It's a lot like an LLM, just slower and considerably more expensive.

Also, you severely underestimate the ability of the authors to edit and verify the correctness of their own work. They wrote it in the first place, after all. They know, better than anybody else, what's important to convey.

What's more, this editing step happens before peer review, so if the paper is junk or has glaring errors it'll probably be turned down by most respectable publications. Even the marginal publications won't publish stuff that's really bad.

raincole• 1 year ago

In my (non-English-speaking) country, paying for an editing service before one submits a paper was very common. Now it's all AI edited.

That's not the interesting part though. The interesting part is that the biggest editing service provider here explicitly states it's okay for the clients to send the papers through AI first then buy their service for 'additional human editing.' They even offer cheaper price for papers that are already edited by AI.

The irony is not lost on me.

jug• 1 year ago

Don't LLM's have enough insight in at least the very concepts here? o3 is considered PhD level on this. Maybe not pure math solving since LLM's suck on that, but I wouldn't be quick to doubt the capacity of understanding the advanced concepts and typical wording. Also, this is hopefully not passing through LLM's completely unvetted, but as a crutch to get 80-90% there with minimal effort.

perching_aix12• 1 year ago

Sounds a bit scary from a correctness standpoint. I have to wonder if such paper authors have the necessary language skills to ascertain whether their LLM of choice didn't e.g. add in any nuance that isn't correct or intended.

I do think it's possible they do, I myself am a foreign speaker who would consider doing something like this, and I think I could notice if that happens. But then I think I'm also well beyond the level where this would be a necessity for me, rather than just a convenience and an additional polishing step.

diggan5• 1 year ago

Wouldn't it be the same if they use the "Author Services" as with an LLM? Essentially "How do you evaluate something that might go above your head?" remains the same regardless of the tool/service used.

mihaaly• 1 year ago

I believe both arguments assume honest contributors, which may not be completely true. I have no reference but vague recollection of doctored data, made up results (basically cheating) due to publication pressure or just dishonesty/lazyness plaguing the reliability of scientific publications. Some may even be happy of the twisted reality or halucinations producing novel sounding results advancing the scientific career similar to those occured in the past. Reproducibility crisis is their friend to get away with it for a long enough time to have the personal gain harvested.

dev_l1x_be• 1 year ago

Absolutely. I think, even for non scientific journals the LLMs have a net good effect on expressiveness and clarity when instructed like that.

einpoklum• 1 year ago

LLMs are masterful translators of language and concepts

No, they are not. They will - speaking generally - try to produce what seems like a typical translation; which is often the same thing, but not always.

diggan5• 1 year ago

They're pretty good at translation, to be honest. Transformers was initially invented/discovered in order to build something better for doing translations with Google Translate, so it's hardly a surprise they are pretty good at it.

fallinditch2• 1 year ago

I both like and dislike the writing style of LLMs.

I like the clarity and easy-to-read usability, but at the same time I find the stylistic tics and overusage of words and phrases often grates.

I wonder if this stylistic failure is something that will get fixed or if it's the inevitable consequence of LLMs.

sandspar2• 1 year ago

It might be because millions of people interact with the same entity and then swap notes. Someone points out "delve" and “it’s not just X, it’s Y,” and other people go, “Yeah, I see that too!”

It's the same thing with celebrity impressions. Everyone’s watched Jack Nicholson enough to sense his quirks without naming them. Then a comedian highlights those quirks, and we’re like, “Haha, that’s EXACTLY it!”

leakycap24• 1 year ago

You need to prompt the LLM to provide the style you want. I have sometimes drag a few of my rough notes into the prompt and tell it to write me an article explaining this topic in the style of a site like The Verge, Daring Fireball, NYT, the onion, etc...

I don't publish this kind of output or build on it, but it makes my own topic I need to review more interesting and makes me question it as if I was reading it on a site like those I mention.

Using this kind of output as your own work seems weird, given it is templating so hard on your references -- but I don't like reading the pollyanna GPT speak.

pona-a• 1 year ago

I think a lot of these are artifacts of RLHF and left-to-right thinking, and one might actually go away with diffusion LLMs. If anyone here spoke to Gemini Diffusion, does it have different speech patterns?

aDyslecticCrow• 1 year ago

First time I've heard about Gemini Diffusion. I thought it was only a matter of time before others would adopt it as soon as I saw the paper by inceptionlabs.ai.

Not sure I'm happy about LLMS getting better... but it feels like an obvious improvement in both computational efficiency and training method.

randycupertino• 1 year ago

At work this week I clocked one of the chief cardiologists I work with using chat GPT for comparing literature around the ideal temperature and fixation time for finding amyloid deposits in skin biopsy samples. He is an arrogant, dry, humorless type who is always grumpy and known for being a pain in my ass and incredibly brilliant and suddenly I have an email from him laden with emojis before bullet points and "It's not just blah - it's blah" writing style. It was classic ChatGPT explanation style and I KNOW it's like that because sometimes I ask ChatGPT to explain confusing work concepts to me and it does the same exact layout with the same emojis before bulleted sections.

It actually made me enjoy him a little bit more because normally he is huge arrogant jerk so it gave me pleasure to see him have a human foibles of relying on chatgpt for his output.

fallinditch2• 1 year ago

Haha great story! - interesting how ChatGPT appears to be softening or improving his communication style, an example perhaps of how LLMs can provide a kind of baseline exemplar for good communication, especially useful for the arrogant jerks.

leakycap24• 1 year ago

Generated text seems to work because so few people read and even fewer think as they read.

AI writing often creates a firehose output of sing-songy, long, over styled text -- which is a lot to ask a reader to get through and really comprehend.

I'm guilty of forming my own conclusion or just believing what I see when I'm in a hurry or unfamiliar or don't care a ton. I'd rather read a human's rough output than a polished generated text version.

Animats3• 1 year ago

Amusingly, the article talks about "improving equity in science". The linked article, "AI tools can improve equity in science" has a link to Google Scholar.[1] The summary there begins "The global space industry is growing rapidly—the number of satellites in orbit is expected to increase from 9000 today to over 60,000 by 2030 . In addition, it is estimated that more than 100 trillion untracked pieces of old satellites ...". That's because Google Scholar hit a paywall and treated the following unrelated letter as the content.

So it looks like the authors used some tool to generate plausible citation links, which they did not read before publishing.

They did some real work. Here's their list of "excess style words", ones whose frequency has increased substantially since LLMs:

accentuates, acknowledges, acknowledging, addresses, adept, adhered, adhering, advancement, advancements, advancing, advocates, advocating, affirming, afflicted, aiding, akin, align, aligning, aligns, alongside, amidst, assessments, attains, attributed, augmenting, avenue, avenues, bolster, bolstered, bolstering, broader, burgeoning, capabilities, capitalizing, categorized, categorizes, categorizing, combating, commendable, compelling, complicates, complicating, comprehending, comprising, consequently, consolidates, contributing, conversely, correlating, crafted, crafting, culminating, customizing, delineates, delve, delved, delves, delving, demonstrating, dependability, dependable, detailing, detrimentally, diminishes, diminishing, discern, discerned, discernible, discerning, displaying, disrupts, distinctions, distinctive, elevate, elevates, elevating, elucidate, elucidates, elucidating, embracing, emerges, emphasises, emphasising, emphasize, emphasizes, emphasizing, employing, employs, empowers, emulating, emulation, enabling, encapsulates, encompass, encompassed, encompasses, encompassing, endeavors, endeavours, enduring, enhancements, enhances, ensuring, equipping, escalating, evaluates, evolving, exacerbating, examines, exceeding, excels, exceptional, exceptionally, exerting, exhibiting, exhibits, expedite, expediting, exploration, explores, facilitated, facilitates, facilitating, featuring, formidable, fostering, fosters, foundational, furnish, garnered, garnering, gauged, grappling, groundbreaking, groundwork, harness, harnesses, harnessing, heighten, heightened, hinder, hinges, hinting, hold, holds, illuminates, illuminating, imbalances, impacting, impede, impeding, imperative, impressive, inadequately, incorporates, incorporating, influencing, inherent, initially, innovative, inquiries, integrates, integrating, integration, interconnectedness, interplay, intricacies, intricate, intricately, introduces, invaluable, investigates, involves, juxtaposed, leverages, leveraging, maintaining, merges, methodologies, meticulous, meticulously, multifaceted, necessitate, necessitates, necessitating, necessity, notable, noteworthy, nuanced, nuances, offering, optimizing, orchestrating, outlines, overlook, overlooking, paving, persist, pinpoint, pinpointed, pinpointing, pioneering, pioneers, pivotal, poised, pose, posed, poses, posing, predominantly, preserving, pressing, promise, pronounced, propelling, realm, realms, recognizing, refine, refines, refining, remarkable, renowned, revealing, reveals, revolutionize, revolutionizing, revolves, scrutinize, scrutinized, scrutinizing, seamless, seamlessly, seeks, serves, serving, shaping, shedding, showcased, showcases, showcasing, signifying, solidify, spanned, spanning, spurred, stands, stemming, strategically, streamline, streamlined, streamlines, streamlining, struggle, substantiated, substantiates, surged, surmount, surpass, surpassed, surpasses, surpassing, swift, swiftly, thorough, transformative, typically, ultimately, uncharted, uncovering, underexplored, underscore, underscored, underscores, underscoring, unexplored, unlocking, unparalleled, unraveling, unveil, unveiled, unveiling, unveils, uphold, upholding, urging, utilizes, varying, versatility, warranting, yielding.

That's the vocabulary of business hype. Those are words that appear frequently in company press releases, and are copied into business publications by cut-and-paste journalists. LLMs asked to polish scientific material will use that vocabulary, even in scientific writing. If you trained an LLM on scientific publications, you'd get more scientific jargon, probably misused.

[1] https://scholar.google.com/scholar_lookup?title=AI+tools+can...

userbinator2• 1 year ago

"improving equity" as in "dumbing down everyone", indeed that's what LLMs have done.

leakycap24• 1 year ago

Good tools can be used incorrectly.

I wouldn't want to go back to a world without LLMs, but I wish people were more thoughtful in how we use them. Right now, we're seeing AI being slapped on everything like every other trend in the past. When things settle after this upheaval, whether in 20 years or 100, I'd bet AI will still be here and be more useful than ever.

tempodox• 1 year ago

That's the vocabulary of business hype.

And I don't think that's a coincidence either. “Do not trust any statistics you did not fake yourself”, do not trust an LLM you did not train yourself.

rawgabbit• 1 year ago

These words are what George Orwell ranted against in https://www.orwellfoundation.com/the-orwell-foundation/orwel....

Orwell said these words are vague and are used to obfuscate. Typically to defend the indefensible.

Animats3• 1 year ago

Yes, that's well known. Orwell on language is an important subject too long to discuss here. His rules, though, are still useful:

i. Never use a metaphor, simile or other figure of speech which you are used to seeing in print.

ii. Never use a long word where a short one will do.

iii. If it is possible to cut a word out, always cut it out.

iv. Never use the passive where you can use the active.

v. Never use a foreign phrase, a scientific word or a jargon word if you can think of an everyday English equivalent.

vi. Break any of these rules sooner than say anything outright barbarous.

(Much of this came from Orwell's work during WWII, when he worked at the British Ministry of Information. Part of his job was translating news into 1000-word vocabulary Basic English for broadcast in the Colonies, mostly India. Hammering political speech down into Basic English requires forcing it into concrete terms, which is an act of political interpretation. That's where Newspeak, in 1984, came from. 1984 is semi-autobiographical, based on his WWII job. See "Orwell, The Lost Writings". One of his managers was called "Big Brother" behind his back.)

foxglacier2• 1 year ago

The actual DOI link also goes to that space article on science.org. It could be just that we as well as Google only sees that but the authors saw the real paper. Of course it's part of a big load of cite-spam so even if they did read it, they probably just cherry picked something - anything that seemed to support their vague low value claim.

Animats3• 1 year ago

Right.

They discovered something real. LLMs turn scientific papers into business-speak, and that there's a lot of that going on. Then they tried to dress up that result for publication, ineptly. Doing so probably improved their chances of getting published.

The word list I posted above is from their additional data PDF file. The paper itself doesn't contain it. Once you see their word list, it's obvious to anyone who follows business news what's going on. Scientific papers are being run through a process that converts them to press releases.

That's a good result, made worse by running it through the process the paper derides.

sandspar2• 1 year ago

File under "LLMs as Western soft power victory". Chinese AI's have Western liberal sensibilities, English achieves even deeper dominance in scientific publications etc.

leakycap24• 1 year ago

It seems like calling a race the second the light turns green. If anything, recent events including AI show me that monumental shifts are more possible today than ever before.

pcrh5• 1 year ago

As a researcher myself, I find it curious that so many of my peers are turning to LLMs to assist in writing.

Writing is a process of communicating one's data and thoughts, and it is the coherent crystallization of such thoughts that produces good scientific papers.

I can't imagine a scenario where an LLM would somehow be able to articulate the outcome of original research better than the researcher themselves. And if the researcher can't articulate their findings, it's dubious that they are worth communicating.

The exception would only be those who are not native English writers, in which case the LLM might improve grammar, etc.

gapan2• 1 year ago

The exception would only be those who are not native English writers, in which case the LLM might improve grammar, etc.

So, only just the vast majority of people in this world? I'm not sure I would call that "the exception".

pcrh5• 1 year ago

An LLM would only be able to "polish" a non-native speaker's writing. Otherwise it is not conceivable that it could improve the essential nature of original research being communicated, this is as said original research will not be part of its training dataset.

Scientists who hope that using an LLM to help write a paper will improve the quality of their output are deluding themselves.

jeffbee• 1 year ago

Paper has an A+ title that really should have been submitted!

arbirk• 1 year ago

I always take issue with the saying the llms are biased. 1) because they are obviously biased as per their training and instructions 2) but are they more biased (at the moment) than google search and wikipedia?

They are tools and we all have to get used to them.

Soon grok and Deepseek will probably be biased when their owners crack the code, so citing what tools you use will be important.

bello33595985• 1 year ago

I need to get full current network

LLM-assisted writing in biomedical publications through excess vocabulary

Related Stories