Reddit Search: A search engine for Reddit indexable by user, subreddit, and date

https://camas.github.io/reddit-search/

by lawrenceyan • 5 years ago

86 40 5 years ago

camas.github.io

It’s a real pity Reddit plays so many tricks with its content: from preventing search engines to index it to flat out editing comments.

hardwaresofton• 5 years ago

Can you go into this? I'm vaguely aware of reddit having changed in this way but not super sure which rules of this ilk are now implace.

This must be why they're login-blocking some big subreddits now, the web scraping precedent that went against LinkedIn must let them know that they can't build a walled garden without making people login first.

CodesInChaos• 5 years ago

Since they only do that on the mobile website, I'd assume this is just another way of wanting to track users and push them to the mobile app.

Dma54rhs• 5 years ago

They don't want you to use their mobile website, I don't remember any site being more hostile and anti user friendly even after saying 100 times I'm OK using browser, not an app.

latentdeepspace• 5 years ago

How is this different from https://redditsearch.io ? If I understand it right, both uses the same PushShift API and the rest is just UI.

mattowen_uk• 5 years ago

Ok this is interesting... A few months ago, I deleted all my Reddit comments and posts (one by one, using a bit of web automation), yet they all still show up in this search even thought they don't on Reddit.

Proves the point, that once something is on the internet, it's there forever.

doix• 5 years ago

Yep, pushshift indexes all reddit comments as they come in. Unless you delete your comment fairly quickly, it'll be stored forever.

stefan_• 5 years ago

So it is a big copyright violation machine? I give reddit a license, not "pushshift".

nabakin3• 5 years ago

Not forever. I've seen it said on reddit that you can email the dev or pm him on reddit and he will remove all of your data. Of course since these archives have been up for download, your data will never truly be gone but it does remove you from the source of these downloads and all future downloads.

rK5Q58• 5 years ago

I've had Shreddit[1] on a weekly cronjob for the past 3 years and i think most if not all of them still shows up here...

[1] https://github.com/x89/Shreddit

nabakin3• 5 years ago

Be careful. Shreddit does not get rid of all reddit comments, only the ones on your profile. Reddit stores a list of the last 1000 comments and posts you've made to your profile and that list is what Shreddit and 99% of reddit 'deletion' tools use. To truly remove all your comments and posts from reddit, look into reddit-shreddit [0]. Despite the similar name, it is completely different and leverages all your posts and comments from a reddit data request to delete all of your reddit posts and comments. It is infact the only method I have found, except for perhaps a GDPR request, that is truly a complete deletion.

However, this only covers the deletion from reddit itself. There have been periodic reddit archivals ever since it was created. Up to 2015 they were done yearly, ever since they have been done monthly. Since they have always been up for download, all of your reddit data will still be out there, however you can remove your data from the main source of these archives called pushshift. I haven't done it personally but I've seen it said on reddit, you can email the dev of pushshift or dm him on reddit and he will remove your information from his archive.

[0] https://github.com/nixfu/reddit-shreddit

ceilingcorner• 5 years ago

I can’t tell if this displays deleted comments, but if the other comment is correct, it does.

That’s a good thing, as I’ve found over-active moderation to be a serious problem on certain subreddits. Especially “Ask” ones like /r/askhistorians. So many insightful or helpful comments are deleted because a trigger-happy moderator arbitrarily decides they don’t “fit the rules,” which of course are flexible for some but not others.

INTPenis• 5 years ago

I will give this a try.

Can someone please tell me why does the reddit search engine default to searching globally? 99 out of 100 searches I make I am on a subreddit, want to search the subreddit, enter my search term and the first thing it does is search globally. I have to click the link r/subreddit to make it search locally.

Most times the API doesn't even have time to return the results before I click the link and make it do another request to search locally. Isn't that wasteful?

I am of course talking about the new UI.

optimalsolver• 5 years ago

It's up to the subreddit's mods whether the search bar does a local or global search.

It's obviously in Reddit's interest for users to be able to search site-wide, but mods can change this behavior.

Seems a fair setup to me.

thatguy09002• 5 years ago

Very strange to put that in the mods control. Why would the mods have any idea or care for what users search for? Reddit has all the data about what users usually want and users know what they actually want, I don't see where mods come in

ketralnis• 5 years ago

Is it? I'm pretty sure they just remember the setting from last time you searched, I don't see this setting anywhere in the subreddits I mod

apple4ever2• 5 years ago

That drives me BONKERS and they need to fix it.

gilbraithVO• 5 years ago

God knows there needs to be a better way to search reddit

drexlspivey• 5 years ago

I just use google with site:reddit.com

tediousdemise2• 5 years ago

Adding site:reddit.com is a little too verbose for me, so ultimately I end up just appending/prepending "reddit" to my query and it seems to do well enough for most of my searches.

asdev• 5 years ago

I built this small chrome extension to add how long ago a reddit post was made, number of upvotes and comments to Google search results since I was sick of clicking into super old threads or threads with no comments:

https://chrome.google.com/webstore/detail/reddit-search-help...

unfortunately the friction of using another search engine is too much for me, this extension helps a little bit at least

brw• 5 years ago

I didn't even realize I needed this but that's actually insanely useful. Thanks!

tediousdemise2• 5 years ago

Does anyone else append or prepend "reddit" to most of their Google searches?

I mostly do it to see real feedback from real people about whatever I'm interested in, rather than an ad-sense driven list of search results organized in descending order by the highest paying company.

speedgoose• 5 years ago

I do. I also add news.ycombinator.com when it's about technology. It provides better discussions than reddit on average.

willio58• 5 years ago

Not most by any means but if I’m looking for a product, I prepend Reddit to see real people talking about it.

jasonladuke0311• 5 years ago

Nearly everything I search for these days.

ecf• 5 years ago

I used to do this a whole lot more until Reddit started screwing with the dates of their posts. Now trying to find something within the last year will see results from five years ago show up.

Additionally, Reddit realizes they are being used this way and have made great strides in building out the astroturfing side of their business to allow brands to post ads anonymously.

apple4ever2• 5 years ago

I do as well - I use Bing but I experience the same thing. Any search for advice type things is met with a bunch of worthless ad driven junk. Adding Reddit at least gets me something worthwhile.

kgin• 5 years ago

Almost always

BayAreaEscapee• 5 years ago

This is really quite an interesting search engine. I was able to find posts by people who had deleted their accounts by using their deleted username in the "author" field.

It's a good reminder to watch what you say on the internet. Nothing ever really disappears.

kQq9oHeAz6wLLS2• 5 years ago

It's a good reminder to watch what you say on the internet. Nothing ever really disappears.

That's why I always use my real name online; keeps me from saying something I wouldn't say in public.

codezero• 5 years ago

Nice to meet you, how do you pronounce your name?

nabakin3• 5 years ago

I've done a lot of research on deleting reddit data recently and there are some important things the reddit community should know about reddit data and actually deleting it.

The most important thing is that 99% of reddit deletion tools out there (in fact all I have found except for two) do not actually delete your reddit data.

Most of these tools rely on the last 1000 posts and comments listed on your profile but do not delete any other comments or posts you have made and even then, most of these do not edit your comments and posts before deleting which means those 1000 posts and comments are still stored on reddit's servers.

To completely remove your reddit data, I have found only one way and that is with reddit-shreddit [0]. Despite its name being extraordinarily similar to another popular reddit deletion tool called Shreddit [1] which doesn't actually work, it is entirely different. reddit-shreddit works by taking the data from a reddit GDPR data request which includes links to all your posts and comments and editing and deleting each and every one. With this method, reddit-shreddit does not need to scrape posts and comments from reddit to know what to delete. It can leverage the official GDPR-compliant data request from reddit ensuring all of your posts and comments are known and are deleted. If you wish to confirm they are actually deleted, you can submit another data request to verify all of your posts and comments are gone from reddit's servers. It may also be possible to remove your data from reddit with a GDPR request but I'm not sure. I haven't looked into that.

Although you can remove your data from reddit's servers, that does not mean it is gone. Ever since reddit was created, people have been scraping and archiving it. The most popular reddit archiving service is called pushshift and it has archives for every year of reddit up to 2015 when it started archiving every month. Pushshift runs the API that powers this website and other popular reddit tools like removeddit and the former ceddit. Luckily the developer has been known to delete your data if you request it. I've seen it mentioned on reddit that you can email him or pm him on reddit and he will remove your data. Unfortunately all the archives from previous years are still floating around the internet outside of pushshift but at least your data will disappear from the largest source.

[0] https://github.com/nixfu/reddit-shreddit

[1] https://github.com/x89/Shreddit

truth_• 5 years ago

All I wanted was a sort by NSFW, and top.

Reddit Search: A search engine for Reddit indexable by user, subreddit, and date

Related Stories