News publishers limit Internet Archive access due to AI scraping concerns 5 months ago
I’m coming at this from a founder/product angle, not a technical one, so excuse the naive framing.
What worries me isn’t scraping itself, but the second-order effects. If large parts of the web become intentionally unarchivable, we’re slowly losing a shared memory layer. Short-term protection makes sense, but long-term it feels like knowledge erosion.
Genuinely curious how people here think about preserving public knowledge without turning everything into open season for mass scraping.