Ask HN: How to tell the world a sites content may not be scraped for AI?

https://news.ycombinator.com/item?id=36850263

by andrewstuart • 3 years ago

1 5 3 years ago

news.ycombinator.com

If you think the companies scraping the data care at all what your opinion is you're going to be disappointed. It's the forgiveness instead of permission approach. Break the rules and negotiate the costs after you've already trained your model and are raking in the money.

samstave2• 3 years ago

WE NEED A "shorts" model.

an anti-AI that tracks what people are negatively suggesting about AIs...

but this will be hedged by hedgies asap with HFT cards and immense buffers....

And so then you can track the transactions lag and determine those that were being tracked by KFT firms, then pre-long their shorts... (in uT)

PaulHoule• 3 years ago

https://en.wikipedia.org/wiki/Robots.txt

emedchill• 3 years ago

If you don't want someone/something from seeing your content, don't put it on the internet but if that isn't enough:

- add a disallow in your robots.txt (many people say the bots ignore this anyways)

- somehow have your pages so far down in SEO rankings that bots would deem it incorrect/irreverent

- put your content behind a login; this too has it's issues since the bot handler can just get some login credentials to crawl anyways or a user can copy the content elsewhere

- you could also try gaming the system by making your content so offensive that the current AI censorship fad blocks it

- you could try not linking a domain name to the IP, making it harder to find

- sue any AI developer that you think crawled your content

samstave2• 3 years ago

you could also try gaming the system by making your content so offensive that the current AI censorship fad blocks it

recall the /. comment bombs against eschelon in the day whereby they had a signature statement that said all the key-words the pre-snowden-leak folks (myself included) wiould trigger eschelon...

There are a vast number of new keywords, in the AI time-loop, which are really bad to include - but maybe a logic bomb of sorts on AI based on woke triggering is in order...

a signatory paragraph of such words as the F, the N, C, the other F, any pronoun BS, etc... and get AI to be offended based on their requirements to be WOKE in their results...

Ask HN: How to tell the world a sites content may not be scraped for AI?

Related Stories