How we’re approaching AI-generated writing on Medium

https://blog.medium.com/how-were-approaching-ai-generated-writing-on-medium-16ee8cb3bc89

by mfiguiere • 3 years ago

69 93 3 years ago

blog.medium.com

Must be a challenge for those GPT detectors to tell the difference between the typical Medium post and AI generated spam.

dymk• 3 years ago

Good luck enforcing that. I mean that seriously - how would they tell?

whalee• 3 years ago

The arms race between generators and detectors has just begun.

CGamesPlay• 3 years ago

The arms race was first automated in 2014, when the GAN paper was published. That's what GANs do: half the network generates garbage, the other half identifies if it was garbage or real. https://arxiv.org/abs/1406.2661

ClikeX4• 3 years ago

It's essentially the extension of the arms race between writers and plagiarism detectors.

iliane53• 3 years ago

AI content detection is a losing battle imo. Models will become better and people will use the detectors to rephrase the text so it doesn’t get detected.

hunglee2• 3 years ago

we are already seeing plagiarising scams, where scammers copy paywalled human written content, paste into ChatGPT for a tonal rewrite, before republishing the freshly laundered material into your own Substack, in order to grow an audience, which the scammer will no doubt drop his own paywall onto a some point. Free money to be had, hard to see how it can be policed

harvey92• 3 years ago

ChatGPT could log all its output and sell to/share with the likes of TurnItIn. Won't be long before someone offers a zero-log competitor of course.

lm284695• 3 years ago

Take whatever ChatGPT gives you and go to another tool, ask it to "rephrase that text" and you're done.

ClikeX4• 3 years ago

We've already seen Instagram Reels, YouTube Shorts, and TikTok steal content from each other all the time.

Content is a commodity that's losing value, fast. Blogs regularly post meaningless content for the sake of keeping up a regular schedule. I've seen this start years ago when junior devs got recommended to keep up blogs for their resume. The amount of blogposts I've seen documenting basic syntax of programming languages is insane. Or clickbait posts telling you "why you shouldn't use Typescript".

The trend where you need to actively push content for traffic is a bubble that's gonna burst. You're already seeing it happen with streaming services, they're culling their content because it's just becoming way too much.

echelon2• 3 years ago

[...] comment captured a common sentiment: “I’m not interested in my paid subscription subsidizing AI. I signed up to pay humans their worth for doing real work. I don’t want to give AI the eyeball hours or oxygen on a subscription platform.”

We let robots do so much already. The level of Luddite resurgence is through the roof.

I'd absolutely use AI to enhance my work. That doesn't make it any less meaningful.

another_story• 3 years ago

＞ I'd absolutely use AI to enhance my work. That doesn't make it any less meaningful.

I would disagree with this when it comes to many creative works. Things like gallery showings are there as much to sell art as they are to sell the connection with the artist. There's a lot more that goes into the experience of a creative product than just the product itself.

harvey92• 3 years ago

Painters and sculptors have had assistants working with them long before AI came along, but you don't usually meet them at gallery showings either.

ChrisMarshallNY• 3 years ago

I remember an old segment of a news show (It may have been 60 Minutes, I used to watch that, a lot), in the 1980s, that featured a New York "artist." He was a fairly "cliché"-looking chap (Two-tone punk hairdo, whacky sunglasses, obnoxious dress, etc.), and he was quite smug that he didn't paint any of the work attributed to him.

He hired minimum-wage people to do the work, and he spent all his time, schmoozing and going to parties, where he would evangelize it.

Apparently, he did quite well.

As an artist, myself[0], that had struggled to sell anything, I found it rather offensive.

[0] https://littlegreenviper.com/art/Cavalier.png

ClikeX4• 3 years ago

Plenty of musicians have created music using randomised generators since forever. Using AI doesn't have to make the art less meaningful, it depends on what you do with said output.

Plenty of "low effort" art gets valued at arguably ridiculous prices, purely due to the name of the artist. Is suspending buckets of paint on a pendulum to let physics do the work worth more than providing an AI with a prompt?

In the end, the value of a piece of work is in the eye of the beholder.

gitfan862• 3 years ago

It is an emotional reaction. Logically these people know that Photoshop is very powerful without AI and they know that plenty of Artists need Photoshop to do their work, but they are not calling for a ban of Photoshop created art. They are calling for banning AI art because they are afraid

lm284695• 3 years ago

Ah yes of course, we're scared, there are absolutely no moral or ethical questions to ask, no consequences on societies, employment, &c. to plan for

We're just scared so we should shut up and accept whatever is coming as God's truth... the invisible hand and the magic of the market will make everything fall in perfect working order, as it always do, without any external interventions

lancesells• 3 years ago

They are calling for banning AI art because they are afraid

I'm not afraid for myself but I am for the future and the willingness to hand it off creation to these tools. Personally, this is another example of a few deep pockets monetizing and selling back another piece of humanity. All without asking or caring. Facebook monetized your family and relationships. Google monetized the knowledge of humankind.

Right now people using a tool like Photoshop are varying degrees of craftsmen. You could take it away and give them a different tool and they'll be able to achieve something similar. You can take away a laptop from a writer and they'll still be able to write. Someone who illustrate in Procreate could lose their iPad and still achieve something with a pencil.

Now a couple generations down the line the skills will start to be gone and the only creation will be made at $99.99 a month for your OpenAI suite. We've been on the road to idiocracy for awhile and it's just been sped up. I don't think it's a doomsday scenario but I do think it's a potentially uglier future.

ergonaught• 3 years ago

We let robots do so much already. The level of Luddite resurgence is through the roof.

The "This is just Luddites 2.0" sentiments are amusing in a sad way, because they seem to be expressed primarily by people who appear to be positioned well enough to readily understand why this is going to go horribly wrong.

zmgsabst• 3 years ago

The Luddites were right:

Luddites objected primarily to the rising popularity of automated textile equipment, threatening the jobs and livelihoods of skilled workers as this technology allowed them to be replaced by cheaper and less skilled workers.

The industrial revolution made life worse for generations in the early 1800s. We’re already seeing similar regressions — eg, app-driven-jobs and decreased customer service.

And you should look up how the automation of labor turned out for horses.

lm284695• 3 years ago

Yeah I just don't get how people miss this elephant in the room.

Automation as it was sold to us since the 60s is a complete failure on the social aspect. Wages stagnate, we work as much if not more than 40 years ago, retirement age is going up, job security is dead, inequalities are growing, poverty is rising, virtually everywhere in the west.

It's cool to automate factory jobs but if it's to push factory workers into amazon warehouses, uber, e-scooter juicers or food delivery jobs I don't see it as a net positive, capital owners sure generate more capital but that's the last thing I'd like to see increasing.

schnitzelstoat2• 3 years ago

I mean you left out that part where after that it massively improved the standards of living to levels entirely unimaginable prior to the industrial revolution and making prosperity available to all and not just a tiny selection of merchants and aristocrats.

We can hope that the book in AI will lead to something similar - with automation making us far more productive.

sitkack• 3 years ago

Gummy Bears are delicious!

Looking for that new Sunset Magazine issue, Backyard BBQing the Dev.

gareth_untether• 3 years ago

‘Human approved’ would be better than AI assisted.

jcq3• 3 years ago

There always been plagiarism and copies, just that gpt makes it easier and faster... Nothing new just more efficient. So what? Purpose of AI is to accelerate the production of worldwide intelligence, it goes with its downsides like producing more noise. No tradeoff.

YurgenJurgensen• 3 years ago

There has always been war, just that atomic weapons make it easier and faster. Nothing new; just more efficient.

schnitzelstoat2• 3 years ago

Yeah, most of the medium posts about Data Science were people just making a tutorial on how to make a graph in Plotly or how to use kNN in scikit-learn etc.

It's been trash for ages. Now it'll just be even more trash.

It's like someone said - it's called Medium because it's neither rare or nor well done.

phkx• 3 years ago

The formulation of their AI policy is a bit too general. "Created with AI" could apply to both, form and content. Does using a spell/grammer checker count as using AI?

My main concern is with the content, because we're more experienced with the failure modes of humans when interpreting information. I like the other approach shown in the post, which is to cite an AI as you would with any other source. Best include the prompt then, which is also the best way when citing humans. Knowing which model generated the content on what prompt would at least enable some judgement on biases etc. which are present in the response.

Looking forward to the first AI-only interview magazine.

tjpnz• 3 years ago

A typical Medium post might include two (perhaps three, if you're lucky) paragraphs of low quality content and a few snippets of code which were helpfully provided as screenshots instead of text. Punctuate that with some boilerplate about the author's Twitter account and that's pretty much all you're getting.

Maybe I'm generalizing just a tiny bit, but my god, it's full of crap. If I were Medium I would be doing everything I could to encourage AI authored content.

anhner• 3 years ago

so you say it's full of crap, and the solution is for them to allow even more, even crappier crap?

SuoDuanDao• 3 years ago

I think the argument is, AI generated content is less crappy crap than the current crop.

canjobear• 3 years ago

They’ve tried to attract high-quality human work for a while, and it doesn’t seem to have worked for them. Might as well try something new instead of digging in.

BeFlatXIII• 3 years ago

Much like /r/SubredditSimulator, at this point, the robotic crap will stink less than organic feces.

a_c• 3 years ago

There is some truth to it. I personally don't read most blog post, including medium. The quality of content is IMO proportional to the author's experience and their effort in producing the content. On medium/Twitter/whatever blogging platform it is just too easy to create crap, crap that aims to "build up audience". Flood them with AI generated content might hint reader to go somewhere else, hopefully somewhere with more depth.

Book is where I find the most value.

pentagrama• 3 years ago

Don't read on Medium a lot, but about UX design topics, I found some mid to high quality quick readings on this two publications:

- https://uxdesign.cc

- https://uxplanet.org

I will not call it crap.

flakeoil• 3 years ago

I suppose if the text is good and adds value and is factually correct then it does not matter if it's a human or an AI assistant who wrote it. (Except in the cases of plagiarism of course.)

The issue is that many or most of the AI generated texts will not be that good in terms of actual content. There are gaps in the conclusions, gaps in story line, wrong facts etc.

This means there will be a huge load of articles, posts and books out there which do not make much sense, but only possible to notice after having read the text properly as the text still reads fairly well written if just skimming.

It also means that articles need to be reviewed, vetted and scored by humans before we can know if it is worthwhile to read it.

Kind of like it has been up until now, the only difference is that there probably will be such a huge amount of content created which now needs to be reviewed. And that might be a task that is impossible to do.

In the end we might stick even more to reviewed and vetted articles, books etc and be less open to new ideas.

ClikeX4• 3 years ago

I've definitely used AI generated text to help me out with suggestions. It's pretty nice when you have a bit of writers block. All the important stuff like facts, numbers, and code examples should be written by yourself in my opinion.

iliane53• 3 years ago

I suppose if the text is good and adds value and is factually correct then it does not matter if it's a human or an AI assistant who wrote it.

The real issue is the bad content. It’s perfectly acceptable for a human to publish a bad article/blog post but when language models do it it’s more or less spam.

smeej• 3 years ago

Why is it less spammy just because a human wrote it? Plenty of humans produce spammy content.

teddyh• 3 years ago

And how do you trust a reviewer? They might be an AI, too.

tempodox• 3 years ago

We are already drowning in noise, there's too much blogspam and other BS. Now that algorithms can produce orders of magnitude more BS in a fraction of the time, the question of “is this article even worth reading” only gets harder. I'm already suffering from “link fatigue” as it is.

hef19898• 3 years ago

The answer to that is, obviously, another AI reading the AI created content. Should make training of new content AIs so much easier!

epistemer2• 3 years ago

I don't see how Medium or even the concept of shared content has a future.

I am sick of chatGPT in its current form but a future AI assistant with even the slightest bit of personalization will put Medium out of business.

"assistant, show me something interesting" I would think is the future prompt. The slightest customization based on browsing history would put the assistant far beyond anything someone else can show me on Medium or any other platform.

I just can't see articles scored by humans as worthwhile has any future. "Interesting" is not objective or statistical, it is entirely personal.

Most of these currently popular content sites are going to go the way of Myspace. They are just tinkering with myspace music or whatever as if it matters.

xbmcuser• 3 years ago

Looks like people are waking up to the fact that GPT and other AI like it are going to change the economic systems and white collar work. Today most people are just playing around with it but as more people realise its capabilities and start using it as tool that it is and find better ways to use it is going to bring about big social and economic change.

temp00345• 3 years ago

Personally, I don't want to waste my energy reading generated text.

At the core of it, people write in order to transmit some deeply distilled messages about life. It is about sharing the experience of being alive, either as advice or warning about what might happen or will inevitably happen to us. The love, the pain, the emotions, the fear of death, the acceptance - good writing is where we read these things in between the lines, where we feel and empathize with the author and as a result gain some deeper insight which helps us adapt to the ever changing circumstances around us.

There's a lot more encoded in that text than just semantic meaning of words or phrases. Not that all human writers have the 'talent' to encode more than that, but the ones who do manage to shift something inside us.

As of right now, I can feel that a text was generated. Same with images - and with sound. I can't exactly explain it, but it's the same kind of 'plastic' feeling and it's similar regardless of the form (text, image, sound).

I'd really like us to be able to keep this edge over the algorithms, but this might be impossible in the long run.

lm284695• 3 years ago

Personally, I don't want to waste my energy reading generated text

They got you covered, you can vomit the text back into ChatGPT and ask it to summarise it for you ;)

analog31• 3 years ago

You could have a browser plug-in that does this automatically.

itsoktocry• 3 years ago

At the core of it, people write in order to transmit some deeply distilled messages about life.

People write for all kinds of reasons, including this one. Agreed that this case isn't reproducible by machines...but the other 50-80% of writing, including technical writing, might be.

op00to• 3 years ago

I write to share technical details of systems I design or help support. I would love to offload this to a computer.

spaceman_20202• 3 years ago

There are a lot of use cases where I definitely don't mind AI-generated text. Keyword-focused search results, for instance.

If I'm Googling "what is npm" or "what are the five highest mountains in the world", I don't care one bit if the answer is from a human or a machine.

mattpallissard• 3 years ago

I care about it being correct. I might not care who wrote it, but as it turns out who wrote it is a good indicator of correctness.

brookst• 3 years ago

I’d love to see a blind test to see what we really can distinguish. I think I’d detect generated text at better than chance, but I’m not 100% sure.

lloydatkinson• 3 years ago

I can see the ridiculous requirements on some platforms already. To the point where Grammarly is counted as "AI generated" or even Microsoft Office Word spelling and grammar checker have to be identified.

XCSme3• 3 years ago

What is the value that Medium brings to article writers? Exposure? Or most are using it just because they don't want to setup their own blog?

whywhywhywhy• 3 years ago

No idea why anyone would want their writing behind the obnoxious onboarding dialogs Medium has from the reader side.

XCSme3• 3 years ago

I never published on Medium, but one of the benefit that I thought it provided was that it also works as a discoverability platform, so your articles get promoted/pushed to people browsing Medium?

erellsworth2• 3 years ago

I used to publish on Medium. The draw was definitely discoverability. A writer with no following could get their stories added to a Medium publication that had a large following, and build an audience. It was effective. I went from no followers to over 1k in just a few months, and I wasn't even posting that much. When they switched to the paid member model, you could even make a little money off it.

However as time went on Medium started implementing the tyranny of the algorithm, pushing their own recommendations over the stuff you actually subscribed to. Writers, of course, started writing for the algorithm and the quality of stories I was seeing started on a downward spiral. At the same time, for me at least, it became harder and harder to get eyes on my work, which was the only reason to be there in the first place.

XCSme3• 3 years ago

But for blogging it makes more sense to have those sort of platforms link to external blogs/pages.

With YouTube it makes sense, because hosting your own 4k videos and distributing them across the globe is not easy, but with text/blog, it makes more sense to have a directory for (external) blog posts then posting your original content directly on 3rd party website.

jasfi• 3 years ago

This is a good first step. A lot of writers can gain by using ChatGPT, etc, to write outlines and give them ideas. A healthy mix, but then say so, give attribution to the AI as a source.

Overtonwindow• 3 years ago

I don't want to read something written by a computer. I'm sure it's neat and all, but I would be extremely annoyed if I found out an article was not written by a human.

victor106• 3 years ago

It’s a little naive to think that people posting AI generated will be transparent about it.

Unless platforms have software to automatically detect and/or flag AI generated content it won’t help

pixl972• 3 years ago

I suspect that most automated detection will begin to quickly fail as gpt ability improves and the abilities of the worst human writers don't.

imnotlost• 3 years ago

The only real solution to the deluge of content, human or otherwise, is paid curation and moderation.

greenSunglass• 3 years ago

I hate this feeling when I read at the end of the article: This post has been generated by [whatever AI]. Did I waste 5 minutes of my life something that is potentially not real/approximate/not checked? How many of those generated article did I came across and didn't know about it?

jackschultz• 3 years ago

My theory on the way this will progress is the skills of being an editor will become even more valuable than today.

For article writing, or non-fiction book writing even, step one is finding the data, making connections, and planning out the topics and flow, done by a human. In a way, that's how it's currently done, the only difference now might be making sure to format that in the correct way for AI writing generation to handle it.

The words are generated, and then the human editor comes in and corrects issues that might come up, say not good enough connections between topics. This can be fed back to the AI for next time to help improve.

Fictional prose will still be around, as there isn't much better than being able to read and connect with an author, where reading and connecting with AI isn't quite something we can accept (for now).

But non fiction writing is quite mechanical already. If you read the Economist like I do, you'll know what I mean. That writing and connecting of bullet point lists in the order a creator wants can be quite valuable, and having someone do an active edit at the end will be quite valuable as well.

Mockapapella• 3 years ago

This feels like a reasonable solution, though I can’t help but feel a more convenient solution would be to integrate tools directly into the Medium writing experience, and then anybody who uses those tools automatically has their article marked as having used AI assistant technology.

pixl972• 3 years ago

So you're just really giving incentive for the creator to use AI and past the article in the submission box so people that don't want AI articles can't exclude it.

mouzogu• 3 years ago

i barely read human generated content as it is.

kkfx• 3 years ago

I'm more interested in the opposite, or the possibility to finally reach a semantic search that works at web scale.

Processing news to elicit facts is more and more time consuming due to the multiplication of sources and vast propaganda, having something able to crawl and juice contents with a certain training is FAR more interesting than improvement in crappy auto-content-creations to keep flooding the infosphere beyond meteo, earthquakes and other news like that.

How we’re approaching AI-generated writing on Medium

Related Stories