Reddit's robots.txt disallows all web crawlershttps://old.reddit.com/robots.txt by rawfael • 2 years ago 8 1 2 years agoOLold.reddit.com