Anyone got a contact at OpenAI. They have a spider problem

https://mailman.nanog.org/pipermail/nanog/2024-April/225407.html

by speckx • 2 years ago

743 357 2 years ago

mailman.nanog.org

egberts1• 2 years ago

There is always IP filtering, DNS blocking, and HTTP agent screening. Just sayin'.

Before someone tells me to fix my robots.txt, this is a content farm so rather than being one web site with 6,859,000,000 pages, it is 6,859,000,000 web sites each with one page.

apocalyptic0n3• 2 years ago

The reason that bit is relevant is that robots.txt is only applicable to the current domain. Because each "page" is a different subdomain, the crawler needs to fetch the robots.txt for every single page request.

What the poster was suggesting is blocking them at a higher level - e.g. a user-agent block in an .htaccess or an IP block in iptables or similar. That would be a one-stop fix. It would also defeat the purpose of the website, however, which is to waste the time of crawlers

fl7305• 2 years ago

I think he's saying that it's not a problem for him, but for OpenAI?

unnouinceput• 2 years ago

Yup, that's my impression as well. He's just nice to let OpenAI they have a problem. Usually this should be rewarded with a nice "hey, u guys have a bug" bounty because not long time ago some VP from OpenAI was lamenting that training their AI is, and it's his direct quote, "eye watering" cost (the order was millions of $$ per second).

mapmeld• 2 years ago

Based on the page footer ("IECC ChurnWare") I believe this is a site by design to waste time for web crawlers and tools which try to get root access on every domain. The robots.txt looks like this: https://ulysses-antoine-kurtis.web.sp.am/robots.txt

I don't see how this does much to keep bad actors away from other domains, but I can see why they don't want to give up the game for OpenAI to stop crawling.

ajuc• 2 years ago

I mean that's your chance to train the SkyNet, take it :)

dr_kiszonka• 2 years ago

That's an interesting attack vector, isn't it?

throwaway35777• 2 years ago

Single handedly extending the lifespan of all humans by however long it takes them to crawl 6,859,000,000 pages.

mypastself• 2 years ago

Frankly, I didn’t get the purpose of the website at first either. I guess I have an arachnid intellect.

aendruk2• 2 years ago

Arachnid here… What am I looking at? The intent is to waste the resources of crawlers by just making the web larger?

quadrature• 2 years ago

It seems like that, but they're also concerned about the crawlers that they catch in this web. So it seems like they're trying to help make crawlers better ?, or just generally curious about what systems are crawling around.

pvg2• 2 years ago

What am I looking at?

I'd say go ahead and inject it with digestive enzymes and then report findings.

troq133• 2 years ago

You had the sense not to spend the rest of your days crawling it though.

frizlab• 2 years ago

I’d let them do their thing, why not?! They want the internet? This is the real internet. It looks like he doesn’t really care that much that they’re retrieving millions of pages, so let them do their thing…

gtirloni4• 2 years ago

It looks like he doesn’t really care that much that they’re retrieving millions of pages

It impacts the performance for the other legitimate users of that web farm ;)

jeremyjh3• 2 years ago

Some scrapers respect robots.txt. OpenAI doesn't. SP is just informing the world at large of this fact.

cactusplant7374• 2 years ago

The CTO isn't even aware of where the data is coming from (allegedly).

Tubbe• 2 years ago

(admittedly)

whoami_nr• 2 years ago

Thats what the whole thing is about. He is complaining that they don't respect robots.txt

alberth• 2 years ago

Isn't the entire point of these type of websites to waste spider time/resources.

Why do they want to not do that for OpenAI?

ganzuul2• 2 years ago

Might one day come looking for who lobotomized it?

dimask• 2 years ago

And? What are they gonna do about it (apart from making such a person/website momentarily famous).

RobotToaster• 2 years ago

Do androids dream of electric farms?

imnotreallynew• 2 years ago

Isn’t the legality of web scraping still..disputed?

There’s been a few projects I’ve wanted to work on involving scraping, but the idea that the entire thing could be shut down with legal threats seems to make some of the ideas infeasible.

It’s strange that OpenAI has created a ~$80B company (or whatever it is) using data gathered via scraping and as far as I’m aware there haven’t been any legal threats.

Was there some law that was passed that makes all web scraping legal or something?

foobarian• 2 years ago

Why would it not be legal? Was there a law passed that makes it illegal?

brushfoot2• 2 years ago

Web scraping the public Internet is legal, at least in the U.S.

hiQ's public scraping of LinkedIn was ruled to be within their rights and not a violation of the CFAA. I imagine that's why LinkedIn has almost everything behind an auth wall now.

Scraping auth-walled data is different. When you sign up, you have to check "I agree to the terms," and the terms generally say, "You can't scrape us." So, you can't just make a million bot accounts that take an app's data (legally, anyway). Those EULAs are generally legally enforceable in the U.S.

Some sites have terms at the bottom that prohibit scraping—but my understanding is that those aren't generally enforceable if the user doesn't have to take any action to accept or acknowledge them.

bena2• 2 years ago

hiQ was found to be in violation of the User Agreement in the end.

Basically, in the end, it was essentially a breach of contract.

darby_eight4• 2 years ago

Scraping auth-walled data is different. When you sign up, you have to check "I agree to the terms," and the terms generally say, "You can't scrape us." So, you can't just make a million bot accounts that take an app's data (legally, anyway). Those EULAs are generally legally enforceable in the U.S.

They're legally enforceable in the sense that the scraped services generally reserve the right to terminate the authorizing account at will, or legally enforceable in that allowing someone to scrape you with your credentials (or scraping using someone else's) qualifies as violating the CFAA?

withinboredom5• 2 years ago

Most of these SaaS's have a "firehose" that if you are big enough (aka, can handle the firehose), can subscribe to. These are like RSS feeds on crack for their entire SaaS.

- https://developer.twitter.com/en/docs/twitter-api/enterprise...

- https://developer.wordpress.com/docs/firehose/

gtirloni4• 2 years ago

It’s strange that OpenAI has created a ~$80B company (or whatever it is) using data gathered via scraping

Like Google and many others.

bena2• 2 years ago

Scraping is legal. Always has been, always will be. Mainly because there's some fuzz around the edges of the definition. Is a web browser a scraper? It does a lot of the same things.

IIRC LinkedIn/Microsoft was trying to sue a company based on Computer Fraud and Abuse Act violations, claiming they were accessing information they were not allowed to. Courts ruled that that was bullshit. You can't put up a website and say "you can only look at this with your eyes". Recently-ish, they were found to be in violation of the User Agreement.

So as long as you don't have a user account with the site in question or the site does not have a User Agreement prohibiting scraping, you're golden.

The problem isn't the scraping anyway, it's the reproduction of the work. In that case, it really does matter how you acquired the material and what rights you have with regards use of that material.

nutrie• 2 years ago

Scraping publicly available data from websites is no different from web browsing, period. Companies stating otherwise in their T&Cs are a joke. Copyright infringement is a different game.

dspillett2• 2 years ago

The issue often isn't the scraping, it is often how you use the information scraped afterwards. A lot of scraping is done with no reference to any licensing information the sites being read might publish, hence image making AI models having regurgitated chunks of scraped stock images complete with watermarks. Though the scraping itself can count as a DoS if done aggressively enough.

reaperman• 2 years ago

There’s currently only one situation where scraping is almost definitely “not legal”:

If the information you’re scraping requires a login, and if in order to get a login you have to agree to a terms of service, and that terms of service forbids you from scraping — then you could have a bad day in civil court if the website you’re scraping decides to sue you.

If the data is publicly accessible without a login then scraping is 99% safe with no legal issues, even if you ignore robots.txt. You might still end up in court if you found a way to correctly guess non-indexed URLs[0] but you’d probably prevail in the end (…probably).

The “purpose” of robots.txt is to let crawlers know what they can do without getting ip-banned by the website operator that they’re scraping. Generally crawlers that ignore robots.txt and also act more like robots than humans, will get an IP ban.

0: https://www.troyhunt.com/enumerationis-enumerating-resources...

ToucanLoucan• 2 years ago

Also worth noting there's a long history of companies with deep pockets getting away with murder (sometimes literally) because litigation in a system that costs money to engage with inherently favors the wealthier party.

Also OpenAI's entire business model is relying on generous interpretations of various IP laws, so I suspect they already have a mature legal division to handle these sorts of potential issues.

observationist• 2 years ago

The 9th Circuit Court of Appeals found that scraping publicly accessible content on the internet is legal.

If you publish something on a publicly served internet page, you're essentially broadcasting it to the world. You're putting something on a server which specifically communicates the bits and bytes of your media to the person requesting it without question.

You have every right to put whatever sort of barrier you'd like on the server, such as a sign in, a captcha, a puzzle, a cryptographic software key exchange mechanism, and so on. You could limit the access rights to people named Sam, requiring them to visit a particular real world address to provide notarized documentation confirming their identity in exchange for a unique 2fa fob and credentials for secure access (call it The Sams Club, maybe?)

If you don't put up a barrier, and you configure the server to deliver the content without restriction, or put your content on a server configured as such, then you are implicitly authorizing access to your content.

Little popups saying "by visiting this site, you agree to blah blah blah" are not valid. Courts made the analogy to a "gate-up/gate-down" mechanism. If you have a gate down, you can dictate the terms of engagement with your server and content. If you don't have a gate down, you're giving your content to whoever requests it.

You have control over the information you put online. You can choose which services and servers you upload to and interact with. Site operators and content producers can't decide that their intent or consent be withdrawn after the fact, as once something is published and served, the only restrictions on the scraper are how they use the information in turn.

Someone who's archived or scraped publicly served data can do whatever they want with the content within established legal boundaries. They can rewrite all the AP news articles with their own name as author, insert their name as the hero in all fanfic stories they download, and swap out every third word for "bubblegum" if they want. They just can't publish or serve that content, in turn, unless it meets the legal standards for fair use. Other exceptions to copyright apply, in educational, archival, performance, accessibility, and certain legal conditions such as First Sale doctrine. Personal use of such media is effectively unlimited.

The legality of web scraping is not disputed in the US. Other countries have some silly ideas about post-hoc "well that's not what I meant" legal mumbo jumbo designed to assist politicians and rich people in whitewashing their reputations and pulling information offline using legal threats.

Aside from right to be forgotten inanity, content on the internet falls under the same copyright rules as books, magazines, or movies published on physical media. If Disney set up a stall at San Francisco city hall with copies of the Avengers movies on a thumb drive in a giant box saying "free, take one!", this would be roughly the same as publishing those movie files to a public Disney web page. The gate would be up. (The way they have it set up in real life, with their streaming services and licensed media access, the gate is down.)

So - leaving behind the legality of redistribution of content, there's no restriction on web scraping public content, because the content was served intentionally to the software or entity that visited the site. It's up to the server operator to put barriers in place and to make content private. It's not rocket surgery, but platforms want to have their cake and eat it too, with control over publicly accessible content that isn't legal or practical.

Twitter/X is a good example of impractical control, since the site has effectively become useless spam without signing in. Platforms have to play by the same rules as everyone else. If the gate is up, the content is fair game for scraping. The Supreme Court gave the decision to a lower court, who affirmed the gate up/gate down test for legality of access to content.

Since Google and other major corporations have a vested interest in the internet remaining open and free, and their search engines and other tech are completely dependent on the gate up/gate down status quo, it's unlikely that the law will change any time soon.

Tl;dr: Anything publicly served is legal to scrape. Microsoft attempted to sue someone for scraping LinkedIn, but the 9th Circuit court ruled in favor of access. If Microsoft's lawyers and money can't impede scraping, it's likely nobody will ever mount an effective challenge, and the gating doctrine is effectively the law of the land.

Karellen2• 2 years ago

Isn’t the legality of web scraping still..disputed?

Are you suggesting it might be illegal to... write a program that connects to a web server and asks for a specific page, and then parses that page to see which resources it wants and which other pages it links to, and treats those links in some special fashion, differently from the text content of the page?

Especially given that a web server can be configured to respond to any request with a "403 Forbidden" response, if the server determines for any reason whatsoever that it does not want to give the client the page it requested?

cess11• 2 years ago

A similar thing happened in 2011 when the picolisp project published a 'ticker', something like a markov chain generating pages on the fly.

https://picolisp.com/wiki/?ticker

It's a nice type of honeypot.

Octokiddie2• 2 years ago

I'm more interested in what that content farm is for. It looks pointless, but I suspect there's a bizarre economic incentive. There are affiliate links, but how much could that possibly bring in?

gtirloni4• 2 years ago

It'd say it's more like a honeypot for bots. So pretty similar objectives.

Octokiddie2• 2 years ago

So it served its purpose by trapping the OpenAI spider? If so, why post that message? As a flex?

dspillett2• 2 years ago

So, it has worked…

madkangas• 2 years ago

I recognize the name John Levine at iecc.com, "Invincible Electric Calculator Company," from web 1.0 era. He was the moderator of the Usenet comp.compilers newsgroup and wrote the first C compiler for the IBM PC RT

https://compilers.iecc.com/

throw_a_grenade2• 2 years ago

This is honeypot. The author, https://en.wikipedia.org/wiki/John_R._Levine, keeps it just to notice any new (significant) scraping operation launched that will invariably hit his little farm and let be seen in the logs. He's well known anti-spam operative with his various efforts now dating back multiple decades.

Notice how he casually drops a link to the landing page in the NANOG message. That's how the bots will get a bait.

agilob• 2 years ago

It's for shits-and-giggles and it's doing its job really well right now. Not everything needs to have an economic purpose, 100 trackers, ads and backed by a company.

phyzome2• 2 years ago

And yet it has the Amazon links, which makes it appear to have some economic purpose...

schleck8• 2 years ago

The books on there are affiliate links I think.

pflanze• 2 years ago

Linkers & Loaders is their own book (I haven't checked the others).

They have a page at https://www.iecc.com/linker/ where they used to publish a draft of the book contents, but changed the page to say "Chapters were available in an excessive variety of formats, but are not any longer due to chronic piracy", when it got posted to HN at https://news.ycombinator.com/item?id=18424233 and I bundled the files for offline reading. I notified them via email about that asking if they are OK with it but got an unfriendly response that I pirated the files and that wasn't OK, so I took the link down again and they changed that text. (Shrug. I'm not a/the book author, they are. I'll say that I also suggested to them they ask on the page not to do what I did since then I wouldn't have, but they chose their more radical approach.)

euparkeria• 2 years ago

What is a spider in this context?

clarkrinker• 2 years ago

Old name for a web crawler / search indexer

dmd• 2 years ago

https://en.wikipedia.org/wiki/Web_crawler

cosmojg• 2 years ago

A Web crawler, sometimes called a spider or spiderbot and often shortened to crawler, is an Internet bot that systematically browses the World Wide Web and that is typically operated by search engines for the purpose of Web indexing (web spidering).

See: https://en.wikipedia.org/wiki/Web_crawler

btown3• 2 years ago

This reminds me of how GPT-2/3/J came across https://reddit.com/r/counting, wherein redditors repeatedly post incremental numbers to count to infinity. It considered their usernames, like SolidGoldMagikarp, such common strings on the Internet that, during tokenization, it treated them as top-level tokens of their own.

https://www.alignmentforum.org/posts/8viQEp8KBg2QSW4Yc/solid...

https://www.lesswrong.com/posts/LAxAmooK4uDfWmbep/anomalous-...

Vocabulary isn't infinite, and GPT-3 reportedly had only 50,257 distinct tokens in its vocabulary. It does make me wonder - it's certainly not a linear relationship, but given the number of inferences run every day on GPT-3 while it was the flagship model, the incremental electricity cost of these Redditors' niche hobby, vs. having allocated those slots in the vocabulary to actually common substrings in real-world text and thus reducing average input token count, might have been measurable.

It would be hilarious if the subtitle on OP's site, "IECC ChurnWare 0.3," became a token in GPT-5 :)

mort96• 2 years ago

During tokenization, the usernames became tokens... but before training the actual model, they removed stuff like that from the training data, so it was never trained on text which contains those tokens. As such, it ended up with tokens which weren't associated with anything; glitch tokens.

zelphirkalt• 2 years ago

So it becomes a game of getting things into the training data, past the training data cleanup step.

btown3• 2 years ago

It's interesting: perhaps the stability (from a change management perspective) of the tokenization algorithm, being able to hold that constant, between old and new training runs was deemed more important than trying to clean up the data at an earlier phase of the pipeline. And the eventuality of glitch tokens was deemed an acceptable consequence.

aidenn03• 2 years ago

I wonder how much the source content is the cause of hallucinations rather than anything inherent to LLMs. I mean if someone posts a question on an internet forum that I don't know the answer to, I'm certainly not going to post "I don't know" since that wouldn't be useful.

In fact, in general, in any non one-on-one conversation the answer "I don't know" is not useful because if you don't know in a group, your silence indicates that.

abracadaniel• 2 years ago

That’s a good observation. If LLMs had taken off 15 years ago, maybe they would answer every question with “this has already been asked before. Please use the search function”

darby_eight4• 2 years ago

With Wittgenstein I think we see that "hallucinations" are a part of language in general, albeit one I could see being particularly vexing if you're trying to build a perfectly controllable chatbot.

shkkmo• 2 years ago

In fact, in general, in any non one-on-one conversation the answer "I don't know" is not useful because if you don't know in a group, your silence indicates that.

This isn't true. There are many contexts where it is true but it doesn't actually generalize they way you say it does.

There are plenty of cases where experts in a non-one-on-one context will express a lack of knowledge. Sometimes this will be as part of making point about the broader epistemic state of the group, sometimes it will be simply to clarify the epistemic state of the speaker.

digging5• 2 years ago

I wonder how much the source content is the cause of hallucinations rather than anything inherent to LLMs

I mean, it's inherent to LLMs to be unable to answer "I don't know" as a result of not knowing the answer. An LLM never "doesn't know" the answer. But they'll gladly answer "I don't know" if that's statistically the most likely response, right? (Although current public offerings are probably trained against ever saying that.)

CuriouslyC• 2 years ago

A lot of LLM hallucination is because of the internal conflict between alignment for helpfulness and lack of a clear answer. It's much like when someone gets out of their depth in a conversation and dissembles their way through to try and maintain their illusion of competence. In these cases, if you give the LLM explicit permission to tell you that it doesn't know in cases where it's not sure, that will significantly reduce hallucinations.

A lot more of LLM hallucination is it getting the context confused. I was able to get GPT4 to hallucinate easily with questions related to the distance from one planet to another, since most distances on the internet are from the sun to individual planets, and the distances between planets varies significantly based on their locations in cycle. These are probably slightly harder to fix.

hiatus• 2 years ago

Contrast with Q&A on products on Amazon where people routinely answer that way. I have flagged responses saying "I don't know" but nothing ever comes of it.

philipswood3• 2 years ago

I've wondered if one could train a LLM on a closed set of curated knowledge. Then include training data that models the behaviour of not knowing. To the point that it could generalize to being able to represent its own not knowing.

Because expecting a behaviour, like knowing you don't know, that isn't represented in the training set is silly.

Kids make stuff up at first, then we correct them - so they have a way to learn not to.

Rebelgecko• 2 years ago

Reminds me of a joke

Three logicians walk into a bar. The bartender says "what'll it be, three beers?" The first logician says "I don't know". The second logician says "I don't know". The third logician says "Yes".

troq133• 2 years ago

"I wonder how much the source content is the cause of hallucinations rather than anything inherent to LLMs."

Probably true, but if you have quality, organized data, you will just want to search the data itself.

singingfish7• 2 years ago

I'm really cross that the word "hallucination" has taken off to describe this as it's clearly in incorrect word. The correct word to describe it is "confabulation", which is clinically more accurate and a much clearer descriptor of what's actually going on.

https://en.wikipedia.org/wiki/Confabulation

Traubenfuchs• 2 years ago

In fact, in general, in any non one-on-one conversation the answer "I don't know" is not useful because if you don't know in a group, your silence indicates that.

Only a response makes it clear one has read and acknowledged the question and sometimes there are people expected to know, if they don‘t the should say so.

Karellen2• 2 years ago

More glitch token discussion over at Computerphile:

https://www.youtube.com/watch?v=WO2X3oZEJOA

qeternity• 2 years ago

GPT-3 reportedly had only 50,257

The most common vocabulary size today is 32k.

RcouF1uZ4gsC• 2 years ago

R's, John

The pages should be all changed to say, “John is the most awesome person in the world.”

Then when you ask GPT-5, about who is the most awesome person in the world…

anonymousDan3• 2 years ago

Honeypots like this seem like a super interesting way to poison LLM training.

verelo• 2 years ago

What exactly is this website? I don’t get it…

mobilemidget• 2 years ago

search engine trickery to get people to click on his amazone affiliate links I reckon

danpalmer2• 2 years ago

A "honeypot" is a system designed to trap unsuspecting entrants. In this case, the website is designed to be found by web crawlers and to then trap them in never-ending linked sites that are all pointless. Other honeypots include things like servers with default passwords designed to be found by hackers so as to find the hackers.

everybodyknows• 2 years ago

Direct link to its robots.txt:

https://www.web.sp.am/robots.txt

ShamelessC2• 2 years ago

Any data scraped would be instantly deduplicated after the fact by whatever semantic dedupe engine they've cooked up.

anonymousDan3• 2 years ago

What has it got to do with deduplication? I'm talking about crafting some kind of alternative (not necessarily duplicate) data. I agree some kind of post data collection cleaning/filtering of the data before training could potentially catch it. But maybe not!

ezekg3• 2 years ago

Eventually, OpenAI (and friends) are going to be training their models on almost exclusively AI generated content, which is more often than not slightly incorrect when it comes to Q&A, and the quality of AI responses trained on that content will quickly deteriorate. Right now, most internet content is written by humans. But in 5 years? Not so much. I think this is one of the big problems that the AI space needs to solve quickly. Garbage in, garbage out, as the old saying goes.

bevekspldnw5• 2 years ago

The end state of training on web text has always been an ouroboros - primarily because of adtech incentives to produce low quality content at scale to capture micro pennies.

The irony of the whole thing is brutal.

ezekg3• 2 years ago

The end state of training on web text has always been an ouroboros

And when other mediums have been saturated with AI? Books, music, radio, podcasts, movies -- what then? Do we need a (curated?) unadulterated stockpile of human content to avoid the enshittification of everything?

oceanplexian2• 2 years ago

Content you’re allowed and capable of scraping on the Internet is such a small amount of data, not sure why people are acting otherwise.

Common crawl alone is only a few hundred TB, I have more content than that on a NAS sitting in my office that I built for a few grand (Granted I’m a bit of a data hoarder). The fears that we have “used all the data” are incredibly unfounded.

sarah_eu2• 2 years ago

Well it will be multimodal, training and inferring on feeds of distributed sensing networks; radio, optical, acoustic, accelerometer, vibration, anything that's in your phone and much besides. I think the time of the text-only transformer has already passed.

pants2• 2 years ago

OpenAI will just litter microphones around public spaces to record conversations and train on them.

FridgeSeal4• 2 years ago

That doesn’t get around the root problem, just gives us multi-modal junk results lol.

jayd16• 2 years ago

It's true that there will no longer be any virgin forest to scrape but it's also true that content humans want will still be most popular and promoted and curated and edited etc etc. Even if it's impossible to train on organic content it'll still be possible to get good content

RogerL• 2 years ago

Is it (I am not a worker in this space, so genuine question)?

My thoughts - I teach myself all the time. Self reflection with a loss function can lead to better results. Why can't the LLMs do the same (I grasp that they may not be programmed that way currently)? Top engines already do it with chess, go, etc. They exceed human abilities without human gameplay. To me that seems like the obvious and perhaps only route to general intelligence.

We as humans can recognize botnets. Why wouldn't the LLM? Sort of in a hierarchal boost - learn the language, learn about bots and botnets (by reading things like this discussion), learn to identify them, learn that their content doesn't help the loss function much, etc. I mean sure, if the main input is "as a language model I cannot..." and that is treated as 'gospel' that would lead to a poor LLM, but i don't think that is the future. LLMs are interacting with humans - how many times do they have to re-ask a question - that should be part of the learning/loss function. How often do they copy the text into their clipboard (weak evidence that the reply was good)? do you see that text in the wild, showing it was used? If so, in what context "Witness this horrible output of chatGPT: <blah>" should result in lower scores and suppression of that kind of thing.

I dream of the day where I have a local LLM (ie individualized, I don't care where the hardware is) as a filter on my internet. Never see a botnet again, or a stack overflow q/a that is just "this has already been answered" (just show me where it was answered), rewrite things to fix grammar, etc. We already have that with automatic translation of languages in your browser, but now we have the tools for something more intelligent than that. That sort of thing. Of course there will be an arms race, but in one sense who cares. If a bot is entirely indistinguishable from a person, is that a difference that matters? I can think of scenerios where the answer is an emphatic YES, but overall it seems like a net improvement.

bogwog• 2 years ago

Eventually, OpenAI (and friends) are going to be training their models on almost exclusively AI generated content

What makes you think this is true? Yes, it's likely that the internet will have more AI generated content than real content eventually (if it hasn't happened already), but why do you think AI companies won't realize this and adjust their training methods?

loloquwowndueo• 2 years ago

Many AI content detectors have been retired because they are unreliable - AI can’t consistently identify AI-generated content. How would they adjust then?

mlboss• 2 years ago

The only way out of this is robots that can go out in the world and collect data. Write in natural language what they observed which can then be used to train better LLMs.

Salgat• 2 years ago

As long as humans continue to filter out the bad content generated by AI, it should be fine.

eightysixfour5• 2 years ago

It is already solved. Look at how Microsoft trained Phi - they used existing models to generate synthetic data from textbooks. That allowed them to create a new dataset grounded in “fact” at a far higher quality than common crawl or others.

It looks less like an ouroboros and more like a bootstrapping problem.

mattc0m• 2 years ago

AI training on AI-generated content is a future problem. Using textbooks is a good idea, until our textbooks are being written by AI.

This problem can't really be avoided once we begin using AI to write, understand, explain, and disseminate information for us. It'll be writing more than blogs and SEO pages.

How long before we start readily using AI to write academic journals and scientific papers? It's really only a matter of time, if it's not already happening.

FridgeSeal4• 2 years ago

Is this like, the AI equivalent of “another layer will fix it” that crypto fans used?

“It’s ok bro, another model will fix, just please, one more ~layer~ ~agent~ model”

It’s all fun and games until you can’t reliably generate your base models anymore, because all your _base_ data is too polluted.

Let’s not forget MS has a $10bn stake in the current crop of LLM’s turning out to be as magic as they claim, so I’m sure they will do anything to ensure that happens.

TaylorAlexander• 2 years ago

I really really hope that five years from now we are not still using AI systems that behave the way today's do, based on probabilistic amalgamations of the whole of the internet. I hope we have designed systems that can reason about what they are learning and build reasonable mental models about what information is valuable and what can be discarded.

SrslyJosh• 2 years ago

Everyone saying "ouroboros": The phrase you're looking for is "human centipede". =)

mkl• 2 years ago

No, the point is it's a circle, with the output feeding back into the input.

atleastoptimal• 2 years ago

They've obviously been thinking about this for a while and are well aware of the pitfalls of training on AI based content. This is why they're making such aggressive moves into video, audio, other better and more robust ground forms of truth. Do you really think that they aren't aware of this issue?

It's funny whenever people bring this up, they think AI companies are some mindless juggernauts who will simply train without caring about data quality at all and end up with worse models that they'll still for some reason release. Don't people realize that attention to data quality is the core differentiating feature that lead companies like OpenAI to their market dominance in the first place?

FridgeSeal4• 2 years ago

I for one, welcome the junk-data-ouroboros-meta-model-collapse. I think it’ll force us out of this local maxima of “moar data moar good” mindset, and give us collectively, a chance to evaluate the effect these things have our society. Some proverbial breathing room.

altdataseller• 2 years ago

If they follow robots.txt, OpenAI also has a bot blocking + data gathering problem too: https://x.com/AznWeng/status/1777688628308681000

11% of the top 100K websites already block their crawler, more than all their competitors (Google, FB, Anthropic, Perplexity) combined

Jordan-117• 2 years ago

It's not just a problem for training, but the end user, too. There are so many times that I've tried to ask a question or request a summary for a long article only to be told it can't read it itself, so you have to copy-paste the text into the chat. Given the non-binding nature of robots.txt and the way they seem comfortable with vacuuming up public data in other contexts, I'm surprised they allow it to be such an obstacle for the user experience.

lobsterthief• 2 years ago

That’s the whole point. The site owner doesn’t want their information included in ChatGPT—they want you going to their website to view it instead.

It’s functioning exactly as designed.

tivert• 2 years ago

Honestly, that seems like an excellent opportunity to feed garbage into OpenAI's training process.

sangnoir4• 2 years ago

So someone could hypothetically perform a Microsoft-Tay-style attack against OpenAI models using an infinite Potemkin subdomians generated on the fly on a $20 VPS? One could hypothetically use GenAI to create the biased pages with repeated calls on how it'd be great to JOIN THE NAVY on 27,000 different "subdomains"

aaron695• 2 years ago

The website is 12 years old and explained here -

https://circleid.com/posts/20120713_silly_bing

John Levine is a known name in IT. Probably best know on HN as the author of "UNIX For Dummies"

samsullivan• 2 years ago

With all the news about scraping legality you'd think a multi billion dollar AI company would try to obfuscate their attempts.

TechDebtDevin• 2 years ago

If you're not walling off your content behind a login that contains terms that you agree to not scraping, then, scraping that site is 100% legal. Robots.txt isn't a legal document.

hermannj314• 2 years ago

I frequently respect the wishes of other people without any legal obligation to do so, in business, personal, and anonymous interactions.

I do try to avoid people that use the law as a ceiling for the extension of their courtesy to others, as they are consistently quite terrible people.

withinboredom5• 2 years ago

If the industry doesn't self-regulate (ie, following conventional rules and basic human courtesy) ... then it will be regulated by laws.

So let me fix what you said for you:

Robots.txt isn't a legal document, yet.

RIMR• 2 years ago

Robots.txt isn't a legal document.

ROBOTS.TXT is an implied license, just like LICENSE.MD or LICENSE.TXT in any GitHub repo. There are decades of precedent that the ROBOTS.TXT file communicates what is and isn't allowed when scaping web content, and that you should check that file before scraping the rest of the site.

Willfully violating a written license provided in a predictable format absolutely is a civil legal violation. If your license says "You cannot use this to train AI", and an AI company scrapes it up and trains an AI on it anyway, even though you did your due diligence to communicate your terms, then you have a legal right to seek damages if you can prove that they are violating your license.

You're basically arguing that no reasonable web scraper would know about ROBOTS.TXT. That's bullshit, this method of web robot control has existed since 1996. It would be like violating the license terms of a GitHub project, and claiming that you didn't know that the LICENSE.MD / LICENSE.TXT file was a license you were expected to follow...

karaterobot• 2 years ago

My assumption is that OpenAI reads the robots.txt, but indexes anyway; they just make a note of what content they weren't supposed to index.

EVa5I7bHFq9mnYK• 2 years ago

And assign such content double weight in training ..

cdme3• 2 years ago

If they don't respect robots.txt then block them using a firewall or other server config. All of these companies are parasites.

pksebben• 2 years ago

I don't think this message is about "protecting the site's data" quite so much as "hey guys, you're wasting a ton of time and network connect to make your model worse. Might wanna do something 'bout that"

cdme3• 2 years ago

I suppose in that case, let them keep wasting their time.

jeremyjh3• 2 years ago

The entire purpose of this website is to identify bad actors who do not respect robots.txt, so that they can be publicly shamed.

cdme3• 2 years ago

Well, we know where OpenAI lands then.

m3047• 2 years ago

No. I've run 'bot motels myself. I've got better things to do than curating a block list when they can just switch or renumber their infrastructure. Most notably I ran a 'bot motel on a compute-intensive web app; it was cheaper to burn bandwidth (and I slow-rolled that) than CPU cycles. Poisoning the datasets was just lulz.

I block ping from virtually all of Amazon; there are a few providers out there for which I block every naked SYN coming to my environment except port 25, and a smattering I block entirely. I can't prove that the pings even come from Amazon, even if the pongs are supposed to go there (although I have my suspicions that even if the pings don't come from the host receiving the pongs the pongs are monitored by the generator of the pings).

The point I'm making is that e.g. Amazon doesn't have the right to sell access to my compute and tragedy of the commons applies, folks. I offered them a live feed of the worst offenders, but all they want is pcaps.

(I've got a list of 50 prefixes, small enough to be a separate specialty firewall table. It misses a few things and picks up some dust bunnies. But contrast that to the 8,000 prefixes they publish in that JSON file. Spoiler alert: they won't admit in that JSON file that they own the entirety of 3.0.0.0/8. I'm willing to share the list TLP:RED/YELLOW, hunt me down and introduce yourself.)

ta_93902• 2 years ago

I am wondering if amazon fixed the issue or blacklisted *.sp.am

ta_93902• 2 years ago

This can be repurposed as a legal form of ransomware

pay me to shut my absolutely legal site down to make your life easier

TheKarateKid• 2 years ago

Or.. you can not have your bots crawl other people’s property without permission.

layer8• 2 years ago

Content farms of that size should be considered a public order offense and be prohibited.

Animats• 2 years ago

Just feed them bogus info and corrupt their models. That will make them stop.

sandworm101• 2 years ago

Dude, you have caught the spider. Now use it. Start inserting whatever random junk you can until "astronaut riding a horse" looks more like Ronald MacDonald driving a Ferrari.

I feel like inserting "free user-tagged vacation images" into my robots.txt then pointing the spider at an endless series of fabric swatches.

_pdp_• 2 years ago

In the network security world, this is known as a tarpit. You can delay an attack, scan or any other type of automation by sending data either too slowly or in such a way as to cause infinite recursion. The result is wasted time and energy for the attacker and potentially a chance for us to ramp up the defences.

bityard2• 2 years ago

From the content of the email, I get the impression that it's just a honeypot. Also I'm not seeing any delays in the content being returned.

A tarpit is different because it's designed to slow down scanning/scraping and deliberately waste an adversary's resources. There are several techniques but most involve throttling the response (or rate of responses) exponentially.

1317• 2 years ago

He's not done his robots.txt properly, he's commented out the bit that actually disallows it

  # silly bing
  #User-agent: Amazonbot          
  #Disallow: /

  # buzz off
  #User-agent: GPTBot
  #Disallow: /

  # Don't Allow everyone
  User-agent: *
  Disallow: /archive

  # slow down, dudes
  #Crawl-delay: 60

haeffin• 2 years ago

The contents changed between then and now.

disjunct• 2 years ago

Time to link the crawler to a site like keys.lol[1] that indexes and links every Bitcoin private key and figure out a way to sweep it for balances.

[1] https://keys.lol/

dhosek• 2 years ago

Am I the only one who was hoping—even though I knew it wouldn’t be the case—that OpenAI’s server farm was infested with actual spiders and they were getting into other people’s racks?

yreg2• 2 years ago

very xkcd

rubidium• 2 years ago

Going way back to the start

nitwit005• 2 years ago

I was hoping that some large set of keywords generated spider images.

mildweed• 2 years ago

Dear Jane,

Can I have my drawing of a spider back then please?

azurezyq• 2 years ago

This reminds me of the binary search tree project on web crawler behavior research. It was a bit old but in really good quality.

http://drunkmenworkhere.org/219

gwbas1c• 2 years ago

Aren't there plenty of reverse proxies you can put a site behind that will throttle this kind of thing?

symlinkk• 2 years ago

Isn’t it funny that all the “worthless” content out there on the internet is actually changing the world. Like how 4chan was mocked as being a cesspit for losers, but now everyone knows memes like Pepe the frog and Wojack from there. And like now this very comment and the billions of other comments on here, Reddit, Twitter, etc that are regarded as a “waste of time” are being used to train multi billion dollar companies to build the most powerful AI the world has ever seen. For free.

The moral of the story here is if you know something valuable, don’t share it online, because then everyone knows it.

danparsonson• 2 years ago

Like how 4chan was mocked as being a cesspit for losers, but now everyone knows memes like Pepe the frog and Wojack from there.

It's possible to hold those two thoughts in mind at the same time.

Rastonbury• 2 years ago

Not sure if including or excluding 4chan in the training set moves the needle in LLM quality. Reddit for sure but 4chan is questionable

latexr• 2 years ago

Sharing it is often the only reason why it has value in the first place. If the authors of Gangnam Style or The Fox or any other silly viral thing hadn’t shared them they would have had zero value.

twothamendment• 2 years ago

I'm so tired of bots. A certain bot from Singapore started pulling all of the product images across multiple domains. Ok, whatever... Then we realized it never stopped. It made enough requests to download them 4-5x over and was still going. The AWS bill was not nice.

We added them to our robots.txt, but traffic didn't stop. I complained to their provider, who happened to also be AWS. Oh, the shock - they didn't care and recommended using robots.txt! AWS was making good money on both ends with this bot that apparently had more money to burn than we do.

EGreg• 2 years ago

This is exactly why we need to move past “the Web” to precisely pay for storage and hosting with micropayments. It is the requester who should pay!

sexy_seedbox• 2 years ago

Use Cloudflare? Just give their server a captcha challenge?

marcosdumay• 2 years ago

Block their origin network...

quesera• 2 years ago

I've been there.

Can't you just 404 the bot in your reverse proxy, optionally setting logging to /dev/null?

Still annoying, but a smaller AWS bill. And maybe the requester will eventually get the point.

xyst• 2 years ago

So that’s where the millions of dollars per day of server/gpu time goes to.

1vuio0pswjnm7• 2 years ago

Anyone care to explain the purpose of Levine's https://www.web.sp.am site. Are the names randomly generated. Pardon my ignorance.

This is the type of stuff the news organisations should be publishing about "AI". Instead I keep reading or hearing people referring to training data with phrases like, "The sum of all human knowledge..." Quite shocking anyone would believe that.

felarof• 2 years ago

If you want to a/b GPT and Claude3 test, try out http://www.clashofgpts.com

Anyone got a contact at OpenAI. They have a spider problem

Related Stories