Google no longer producing high quality search results in significant categories

https://twitter.com/mwseibel/status/1477701120319361026

by lando2319 • 5 years ago

2813 1271 5 years ago

arkitaip2• 5 years ago

It's not just those categories that are producing low quality SERPs, almost any niche you can imagine is being assaulted using weaponized SEO because content marketing has become the competitive advantage of many organisations and part of the default toolkit of any site team.

csomar• 5 years ago

Is it weaponized SEO, though? Because I have been adding "wikipedia" recently to many of my searches. Google has penalized Wikipedia in search results significantly (at least for me). I'm not sure what's the logic behind especially that 1. Wikipedia is trustworthy and ad-free and 2. I click mostly Wikipedia, so from a MLearning perspective I should be getting more wikipedia results.

fnord1233• 5 years ago

Just use ddg. By nowadays its just better.

Guest42• 5 years ago

Same, I tend to trust wikipedia and it hasn't been showing up nearly as much

arkitaip2• 5 years ago

I've been using a Firefox search keyword for Google+Wikipedia and it works great.

eddieh2• 5 years ago

Honestly, I think a good search engine would hardcode the first two results to be the official website for "thing" and the wikipedia page for "thing" if they exist. Then it can list out the organic results. Just that change would make search 10X better than it currently is.

MiddleEndian4• 5 years ago

Consider skipping a step and querying wikipedia directly. Bookmark https://en.wikipedia.org/w/index.php?search=%s&title=Special... in your browser and giving it the shortcut w (or whatever you think makes sense)

paganel• 5 years ago

I found out today that one of my reddit comments (along with many random others) had been scrapped out and automatically put back together (together with those many random other that is) on a blogspot page. I stumbled upon that page today as I was googling for a very specific topic, it went something like this: that SERP looks really interesting, nice! => this page looks like spam => this page is spam => damn, that bunch of text is really mine.

Justin_K• 5 years ago

Their algorithms find it more engaging if I have to click through 50 crappy links to find one good one. If I spend 5 extra minutes browsing Google, that's a great metric, right?

antisthenes2• 5 years ago

Depends if you have adblock or not.

15characterslon• 5 years ago

Maybe deliver high quality results for adblock users only. Lower server costs for adblock users while maximizing ad-revenue for users without adblock. Win-win.

Nextgrid19• 5 years ago

If those crappy links have Google ads or analytics then it's absolutely beneficial for Google.

solarkraft2• 5 years ago

Not only that, there are still ads on the search page.

beebeepka• 5 years ago

No longer you say. That ship sailed more than a decade ago.

I can't know for sure why that is but I'd bet on accurate results not making as much money. It's been like a brochure for a long time

marginalia_nu5• 5 years ago

It's been a long time coming, but it seems to have hit a critical point recently where it's so obviously bad that it's becoming very apparent.

eddieh2• 5 years ago

You're right. I've been complaining about Google results for more than a decade. Things keep getting worse. People used to dismiss my allegations, but I'm seeing more people noticing now days.

pastelsky2• 5 years ago

It's impossible to get accurate information about travel, hotels, locations. Results are often filled with listicle crap that each copy each other and contain outdated pop content.

I've become so used to suffixing these queries with "forum" or "reddit" to get articles written by real humans.

marginalia_nu5• 5 years ago

A funny thing is that the spammers are adapting to this. I found a bunch of websites that all had "reddit" in the title a while back, because that is something people add to filter out the spam.

https://memex.marginalia.nu/pics/reddit-spam.png

daxuak2• 5 years ago

I use `site:reddit.com` instead of `inurl:...` or just reddit.

ncpa-cpl3• 5 years ago

One useful feature I use before visiting new places is sorting Google Maps reviews and photos by date. I've found Google Maps comments more accurate than TripAdvisor or Airbnb.

dleslie2• 5 years ago

You ought to try using duck duck go. It's all about being a portal to other search services.

Ie, searching for `foo` on reddit is: "!r foo"

sefrost2• 5 years ago

The problem with that is it redirects you to the Reddit search page, which isn't great. It's much better for me to use "site:reddit.com [query]" on DDG than to use the !r.

CactusOnFire2• 5 years ago

Duck Duck Go is great at naive searches, though having context into why I'm searching for something can be helpful too.

I couldn't make the switch over for my work while programming.

cpeterso• 5 years ago

Seems like we need a search engine for trusted sites, something like MetaCrawler that searches more specialized search engines. There are some sites I trust for travel information. There are other sites I trust for reviews of consumer products and others for movies.

goodells• 5 years ago

Even appending “reddit” is often inadequate, as the algorithm seems to artificially limit one search result item (potentially with some children under it, but unrelated threads, 2-3 more items) per host. So it becomes necessary to use “site:www.reddit.com” to get more than a few non-sucky results.

deelowe• 5 years ago

I had no idea others have begun resorting to this as well these days. My assumption was that I had just somehow gotten worse/impatient at researching things or that reddit was just a more reliable source these days. Thinking back, may search quality has just gotten wrose.

DSingularity• 5 years ago

Exactly my experience this year. For products I have to watch so many videos. Last year it was easier and all reading.

sidibe• 5 years ago

Reddit is a great place to get recommended the most expensive stuff. 99% of the time the best sellers on Amazon work great for me, but if I look at Reddit they insist only a couple very expensive brands are worth owning in any category of item. It's crazy and sometimes I even fall for it. Can't tell if it's marketing or if people very into their hobbies become obsessed with tiny differences.

neuronic2• 5 years ago

Reddit is full of subversive marketing. The real communities discussing their passions are absolutely exploited by "undercover" PR agencies.

rowanajmarshall• 5 years ago

I find that's true in specialised subs, but not wider ones. Like, if you're looking for a good coffee grinder, r/coffee will only recommend the top-end stuff, but if you search in something r/AskReddit you get more 'approachable' results.

baxuz2• 5 years ago

I've made a custom search in Chrome with the "r" hotkey, that searches Reddit in the past year. Sites are also starting to just fake the dates so I'm getting really old results on regular. This takes care of both:

  https://www.google.com/search?q=%s+site%3Areddit.com&tbs=qdr:y

lifeplusplus2• 5 years ago

This! Had a printer bug and all results where like top 3 ways to share a printer

not2b3• 5 years ago

Suppose a Google competitor emerged that produced wonderful results in every category. Within a week, tens of thousands of SEO specialists will be on the case, reverse-engineering the magic and figuring out how to get their crappy sites back to the top of the rankings. The wonderful results would quickly degrade, and it's unclear as to whether this new small company would have the resources to keep up.

Nextgrid19• 5 years ago

Within a week, tens of thousands of SEO specialists will be on the case, reverse-engineering the magic and figuring out how to get their crappy sites back to the top of the rankings.

The Google competitor will have a human nuke their entire domain or business (based on a manual index of banned products/brands) from the search results forever, or have by default a bias against ad-filled websites which would remove any commercial incentive for those websites to exist in the first place.

ncann2• 5 years ago

That kind of manual intervention doesn't scale though. The only way it can work is to have community-curated lists of bad domains, similar to adblock lists, that users can upload to personalize their search result.

justaguy37• 5 years ago

I hope they'd have a human complaints department too, for when (a real) business is upset

daxuak2• 5 years ago

I wonder if there's a project like sponsorblock, for google search instead of youtube. Basically a centralized community-driven database that blacklists certain urls (timestamps for yt) based on submission & vote. It would be much less of a clear cut than youtube's case, though.

I personally just append `wiki` or `reddit` to queries. Crappy but kinda works.

Nextgrid19• 5 years ago

It doesn't have to be community based. The search engine could be paid and employ actual people to sift through the garbage and vet domains/brands/etc before they are added to the index.

The problem with Google is that their business model is to show you ads (either on their own website or third-party websites embedding Google ads/analytics) and not to provide you quality search results, therefore they have no incentive to combat even the most obvious SEO spam (see Pinterest & image search for an example).

keewee72• 5 years ago

The Google competitors should just permaban all sites that contain affiliate links. That will instantly solve 90% of the problem.

The list of affiliate marketing sites might need to be human curated (just like SponsorBlock) to avoid false positives.

wyre3• 5 years ago

This will block sites like OutdoorGearLab which has great content and trustworthy reviews. I think doing a check on the copy on the SEO website and comparing it to other websites might do the trick though.

Retric5• 5 years ago

SEO isn’t magic. Google decided to deprioritize Wikipedia for example which noticeably degraded results. Where major websites show up has nothing to do with SEO and simply relates to what they think is important.

endisneigh3• 5 years ago

How do you know Google is deprioritizing Wikipedia?

Nextgrid19• 5 years ago

Somehow humans are very good at telling apart SEO crap from legitimate content even without understanding the content or the language itself - SEO crap has some common elements such as ads, affiliate links, a certain page layout, etc.

I remember using an open-source ML model (trained on Buzzfeed article titles) to detect YouTube clickbait based on titles and it worked brilliantly, and that was just downloading some code on GitHub and running it as-is. I'm sure the same could be applied to search results and you could achieve much better quality if you actually put some effort into it.

I very much doubt this is some kind of hard problem as opposed to Google just giving up because their business model doesn't actually incentivize good search results.

bullfightonmars• 5 years ago

It is all about incentives isn’t it? Google gave all the power to these SEO websites by making it difficult to get your content listed as a search result on the first page.

Google could start incentivizing high quality unique content and websites from domain experts, but they have decided they can’t make as much money off of independent publishers as they can from marketing/content farms.

newaccount742• 5 years ago

Suppose a dozen Google competitors emerged that produced wonderful, but slightly different results in every category.

They all use different algorithms, so the SEO specialists can't game them all.

Making crappy sites would maybe no longer be worth it.

keewee72• 5 years ago

Doman-Specific Search Engines is a great idea. I already use PriceRunner to find products I need.

rolph3• 5 years ago

i remember a time when teacher would say form a line, and there would be a chaos of who was getting there first.

This was met with a second instruction, the first five people in line, move to the back of the line, and last five in line move to the front.

it became a trial and error, with the usual kids jostling for position now wanting to be, at the back of the line, then in the middle of the line, hunting for the condition that creates pole position.

the idea that it was the personalities and the value sets, not the position on the line, that triggered the condition, seemingly was too abstract to be deduced. keep in mind this was grade 2.

so may the browser filter the first to the nth SEO spammers in the query results be sent somewhere away from the front of the line until a match for query terms occurs.

tobr2• 5 years ago

What if the search index was much more manually curated? For example, say that you could create a custom search index relevant to your field of expertise, and that other users would rank their results to let the engine know which indices are actually good and for which types of queries. You could still game it, but probably not with traditional SEO techniques.

notriddle• 5 years ago

What if the search index was much more manually curated?

Google would need to not have a monopoly if they wanted to do that. Otherwise, they would be accused of anti-competitive practices (since their policies would be aligned with their policies in other services, they would rarely ban themselves).

marginalia_nu5• 5 years ago

The efficacy of SEO is largely dependent on a search monoculture. They only need to optimize for one set of unknown rules, and that's something that is relatively easy to do well with simple machine learning tools.

Real competition in search is probably the best way of reigning in the SEO sector.

ByteJockey• 5 years ago

Within a week, tens of thousands of SEO specialists will be on the case, reverse-engineering the magic and figuring out how to get their crappy sites back to the top of the rankings.

An algorithm could punish things inherent to SEO optimized sites (ads and tracking). Removing the ability to passively generate money without providing useful content is the key.

Of the top of my head, a system like the following would be a pretty good start:

Scale of 0 to 100 (closer to 100 being higher up on the results).

You are penalized for the following:

- subtract 10 points for any ad, using ublock origin's list as a good starting point (this stacks, it's 10 points off for each link)

- subtract 100 points for google tag manager

- subtract 100 points for the facebook like button

- ... etc for each of the major tracking scripts

This would obviously need to be updated as ad-tech evolves, but it would cut out 90% of the current SEO spam.

Can google do this? No, they have a conflict of interest around placing ads. Somebody else, however, can absolutely do this.

betwixthewires5• 5 years ago

Yeah, it's an arms race. That's the business you enter into when you decide to run a commercial search engine.

To me it seems to be compelling evidence that search engines are not a viable way to organize information on the web, but that's a different topic.

cge• 5 years ago

You can unfortunately see this with DuckDuckGo: people are clearly targeting more than just Google. Python results, for example, are so infested by SEO-specialist spam that searching for standard library functions will return spam above the actual standard library documentation, particularly for more popular functions. Searching for "python datetime.now" or "python json.loads" will both return spam above documentation. This problem heavily impacts anything 'data scientist' spammers see as important as well; it's actually worse on DDG than Google.

What's frustrating is that these often seem to be a handful of domains, like 'geeksforgeeks' and 'towardsdatascience'; for a while, of course, there was also the "gitmemory" spammer who seemed to be able to push out Github results on both DDG and Google. Yet I think Google removed reporting and blacklisting domains from searches long ago, and I think DDG never had it, leaving the only option for removing them client-side scripts and extensions that work poorly. Likewise, no search engine appears to be manually blacklisting them. Yet as you point out, if one did, then the spammers would probably just move to using many domains, which would probably be worse.

Nextgrid19• 5 years ago

Domain age & quantity/breadth of content should be taken into account in ranking.

A fresh domain that suddenly has a ton of content should be viewed with suspicion and downranked as it's likely a spammer (copying GitHub/Stackoverflow/official docs).

Legitimate sites that are starting out shouldn't be affected as they are unlikely to have a ton of content from the start.

Of course, this isn't perfect, but it should take care of the majority of spam copycat sites.

codeisawesome• 5 years ago

I have to wonder just what is the justification for Google to not let users decide what sites they never want to see results from? I'm sure it can't be storage or compute limitations. This one feature alone would make so many people (me included) happier to use the engine.

freediver2• 5 years ago

Not if the search engine is paid. The 'circle of doom' you describe works only in "everything is free" world.

rleigh• 5 years ago

One of the problems here is the effective monoculture.

If there were multiple search engines in use with different ranking algorithms and different search strategies, then "SEO" would be difficult to impossible to achieve because optimising would one search engine would likely be a pessimisation for the others.

We don't just need one competitor, we need many. And they can't just be copycats, they need to do their own thing, and that might well make the cost:benefit of SEO unviable.

rc_mob3• 5 years ago

Ive been using Kagi lately, so far so good. Fingers crossed for more competition.

deadalus• 5 years ago

Yandex is the best when it comes to reverse image search, it almost matches Pimeyes.

Bing is the best when it comes to video search.

Brave is great when searching for controversial or censored topics.

fart32• 5 years ago

Google is best for searching Russian sites automatically (and very poorly) translated to my language that hijack history.back(). I'm getting those a lot lately.

fnord1233• 5 years ago

Tip: add NHS or CDC to the end of any health search to get better results from the NHS or CDC instead of however many pages of grifter sites.

jf• 5 years ago

I also recommend buying a paper copy of the Family Medical Guide by the American Medical Association

ape4• 5 years ago

But shouldn't google's page rank find those pages most reputable and list them first?

ClumsyPilot3• 5 years ago

Or go to NHS official website and search there? What's the point of google in this case?

fnord1233• 5 years ago

Theres no point. But people want to search using their url bar. People don't want to change their default search from Google. I use DDG and !nhs so I'm with you.

JCWasmx86• 5 years ago

SEO is so annoying, I recently searched "Is XYZ healthy?", then just some blogs/lifestyle magazines/whatever showed up, just written in an absolutely verbose manner. I would have expected Yes/No, not 4 paragraphs introduction, some about the advantages, some about the disadvantages and so on.

topicseed• 5 years ago

SEOs don't write all of that for fun. They write it because it ranks well, because Google seems to favor this verbose blurby content.

The second Google clearly favors shorter answers, SEOs will publish just that.

SEOs go where the wind blows.

GeoAtreides• 5 years ago

try and use scholar.google.com to find primary literature about... well, anything.

for example: xyz meta-analysis will return meta studies that summarize other studies about xyz

sadly most studies are behind paywalls and only the abstract is visible, but just copy and paste the DOI into sci hub to get the full text

CamperBob2• 5 years ago

I recently searched "Is XYZ healthy?",

I wouldn't expect that to work any better than the other example someone quoted above, where a search for "nice restaurants" returned restaurants in Nice. Put some more thought into your queries. Like most tools, what you get out of a search engine is largely a reflection of what you put into it.

ncpa-cpl3• 5 years ago

Then there's websites like Scribd, SlideShare, and pdf hosting websites which require accounts or subscriptions have hijacked many results. This has happened even if the original pdfs are still at the source.

cracker_jacks2• 5 years ago

Can someone provide some explicit search queries so we can see the bad examples? Lots of criticism is being doled out in that thread without an actual example to see for myself.

symlinkk• 5 years ago

Try “best kitchen knife set” and compare it to “best kitchen knife set reddit”

cracker_jacks2• 5 years ago

Agree that the reddit is more relevant to me because it has multiple perspectives in a discussion format, but I don't see any particularly bad links with the first search. The first one just has more conventional review sites like GoodHousekeeping and Forbes.

littlecranky6710• 5 years ago

Just researched good/quality crafting printers yesterday. Search results were mostly blogs and crappy websites that offered obviously no insights but were just SEO optimized to direct you to their Amazon affiliate links. Especially sad since those affiliate links to Amazon mostly resulted in "This product is currently not available" sites.

Repeated my search on Youtube to find reviews or unboxing. Most video search results were basically "Youtube SEO" again - the most viewed/top-ranked videos did never show a single actual print run or even the printer available. It was mostly marketing websites turned into video (slowly scrolling/moving over product description or pictures clearly taken from the web). And of course, affiliate links in the description.

The web has become a crappy place to research products as long as money can be made with those through affiliations. I wonder if outlawing affiliate marketing would make the world a better place.

P.S: Whats most ridiculous about my Youtube Printer research experience, the best and most helpful video was a sales video from a home shopping TV station [0], where they actually showed some printing action and handling of one of the models I was interested in.

[0] https://www.youtube.com/watch?v=ytMXgjCReO0

a-dub2• 5 years ago

yeah, had this experience myself while purchasing holiday gifts. amusingly, it made me pine for the days when you could walk into a shop and a knowledgeable salesperson would ask you a few questions and pick the right stuff for you.

it's funny because internet shopping became popular in the face of salespeople becoming corrupt under commission and performance schemes, now a big part of the commerce related internet is worse for pretty much the same reasons.

1over137• 5 years ago

it made me pine for the days when you could walk into a shop and a knowledgeable salesperson would ask you a few questions and pick the right stuff for you.

Uhhh, you can still do that, you know.

kristofferR2• 5 years ago

Rose colored glasses. Workers in stores were the original "affiliate marketers", in a lot of stores (especially electronic ones) workers were/are being paid to push the most expensive products.

Then when you try to pay, you get upsold warranties.

JoshTriplett• 5 years ago

it made me pine for the days when you could walk into a shop and a knowledgeable salesperson would ask you a few questions and pick the right stuff for you.

I still remember being in Best Buy and hearing the salespeople scamming less knowledgeable customers about how much computer they need or how important expensive cables are. I don't think there was ever a time when you could trust electronics store salespeople to sell you "the right stuff for you".

megablast2• 5 years ago

You want someone to pick gifts for your family and friends?

datavirtue4• 5 years ago

"it made me pine for the days when you could walk into a shop and a knowledgeable salesperson would ask you a few questions and pick the right stuff for you."

THAT is retail. The craptastic experiment we see these days, online and in meatspace are scams masquerading as retail. I miss Sears.

cheese_van• 5 years ago

it made me pine for the days when you could walk into a shop and a knowledgeable salesperson would ask you a few questions and pick the right stuff for you.

When you use Google you are walking into their shop. That salesperson is very knowlegeable, but about you.

hinkley• 5 years ago

I use DDG which is backed by Bing and it’s much the same. Product reviews and some how to questions are just a couple pages of SEO garbage.

Google used to measure engagement to try to get feedback on the quality of the links, but perhaps clickbait has broken that as well.

riazrizvi2• 5 years ago

It's a policy problem, there's no easy fix.

Design a system of rules to rank content. Watch as highly incentivized participants work harder than everyone else to game the system to their advantage.

"But I can see these results are obviously bad, surely something can be done?". Leads to System-Amendment#5796 = Gather user feedback to alter results. Leads to highly incentivized users gaming the feedback system.

littlecranky6710• 5 years ago

I wonder if we just would disallow affiliate links in general. I.e. you can still create your own webshop, take orders and forward to someone else (similar to dropshipping) but basically outlaw the process of collecting money/commission just for a single placement of links. Or at least, force those sites to correctly mark themselves as "Advertising".

Nextgrid19• 5 years ago

This assumes the rules are black and white and there's no concept of bad faith.

A system could very well have rules/guidelines and then have humans review & monitor the system and user-submitted complaints for any abuse, and harshly penalize such abuse with a temporary or permanent ban.

It could end up in a situation where it's technically possible to gain a slight advantage by gaming the system but no participant will risk a complete ban and the system ends up working well for everyone.

stefan_• 5 years ago

This sounds more like a Google curse - the inability to do something extraordinarily simple because it can't be sufficiently automated.

Like there is no reason to have a site like "https://gitmemory.cn/" in the index, ever. It is pure and utter spam. Ban the domain.

jsemrau• 5 years ago

I ran into the same problem when building finclout's feedback loop. In my opinion, there is no way around keyword matching for search. As a result SEO is unlikely to be preventable. However, if the feedback can be collected at scale and is actually incentivised (i.e., did this link solve your problem ? ) this might actually work.

PaulHoule5• 5 years ago

In photography I have struggled to choose products, often I have to rely on my experience (e.g. it takes six months to discover that your Epson EcoTank printer makes prints that fade in six months) and specialized forums (DPReviews) You might find lightfastness rankings for some printers at

http://wilhelm-research.com/

if somebody paid them to do the research otherwise you might have rely on heuristic 'pigment based inks are relatively lightfast' and that at least one other person who does similar work, has similar skills, and uses similar method gets good results. That still doesn't help with problems like '90% of instances of this Sigma lens seem to be pretty good but 10% have defective autofocus'.

I find online reviews are close to worthless because there are so many people who don't know how to use gear or have unrealistic expectations. At Best Buy I saw a review of a printer where somebody showed pictures of prints they made where they printed on the wrong side of photo paper and blamed the printer, for instance.

addaon• 5 years ago

At least in the past (haven't checked in years), the Amazon affiliate program rewarded the affiliate for the next purchase made (within a pretty long time window) after clicking an affiliate link, even if it's not of the product linked; so there's no downside to sending you to an arbitrary, unavailable, irrelevant product as long as you're clicking on links when you're in a shopping mode and likely to make a purchase soon.

geerlingguy• 5 years ago

24 hours, any purchases made, and it's still a thing. Though rates are much lower nowadays than they were 3+ years ago.

chongli3• 5 years ago

I am in the exact same quagmire trying to research water softeners and drinking water (RO) systems. I am having to pull fragments of information from many different sites / videos and having to discard large amounts of contradictory and misleading information.

At this point, I’ve had to resort to reading scientific papers on how some of these technologies work (and how some don’t work) just to avoid the bad information. Unfortunately, the more I research the farther I get from making a purchase. This is not where I want to be, however, since the water in my area is very hard and everything gets scaled up all the time. I’m looking to purchase an espresso machine to get into the hobby but I’m not going to drop the money in expensive equipment until I can get access to water which will not damage it in short order.

exhilaration• 5 years ago

I bought my home water filter based on a Consumer Reports recommendation. I feel that's still a reliable source. I think you can sign up for online access for a month to do all your research.

I also really like the Wirecutter but I know not everyone here agrees with me.

xtracto• 5 years ago

Ha! I was searching for the same (water softening systems) for Mexico... the amount of signal to noise ratio is so small in google, that it's mostly unusable.

tomrod3• 5 years ago

It isn't just me noticing this.

I had a project take me into an unfamiliar knowledge space. Up until... 2018? I could have relied on Google to find the meaningful information.

The reason people accepted Google was that it dramatically lowered information acquisition costs through the Internet. That benefit doesn't seem as common now. Back to webrings.

xapata2• 5 years ago

Same for my experience buying a furnace. Science journal articles were my best source of information, because I worried the manufacturers and installers would be biased.

I'm not sure how this could be otherwise. Who's going to pay for someone to stay current on such things and publish an easy to understand summary?

DaftDank• 5 years ago

Check out APEC RO systems.

Fricken2• 5 years ago

You guys had a bad time trying to search things that are at least a little complicated and obscure. I had the same experience trying to find a basic brownie recipe last week.

jansimonek• 5 years ago

If you are reading papers on water softeners you might already know everything this video contains. For me it was a good intro to the topic of good water for making coffee: https://youtu.be/jfElZfrmlRs

replwoacause• 5 years ago

I've been there and it's a hell of a rabbit hole, one that is easy to lose days to. Whatever you do don't fall for the Berkey scam.

trevex• 5 years ago

I unfortunately share your experiences. However in Germany there is a foundation called “Stiftung Warentest” [1], which does independent testing of products. With reviews full of Amazon affiliate links dominating search results, I tend to purchase their tests for ~3 Euros more often than ever.

However this only works for popular product categories, but less so for specialized equipment.

My impression is also that affiliate links hurt consumers in the long run as they reduce the selection of products in reviews or blogs to those the authors can earn money with. This however leaves out potential alternatives. More often than not the winner of product categories (at least those I was researching) of independent tests were not available from sites running affiliate programs. For example a consumer-grade lawn mower from an otherwise professional gardening company or a tent from a Scandinavian brand.

[1] https://en.m.wikipedia.org/wiki/Stiftung_Warentest

cortesoft3• 5 years ago

The US has Consumer Reports, which is similar I think... but yeah, only works for major product categories.

emsy• 5 years ago

I have used their tests in the past and I’ve never regretted a purchase influenced by their tests.

tucosan2• 5 years ago

Sadly, Stiftung Warentest often doesn't have the expertise to properly test many products. I often notice the shortcomings in product tests for products that I know and use.

I was reminded of this with their last test of 3d printers. Their test results where far from what everyone with experience in the field would consider accurate.

kkjjkgjjgg• 5 years ago

They are OK, but I remember when they decided a Nokia phone was better than an iPhone.

Tests usually have subjective criteria that determine the ranking.

amelius2• 5 years ago

which does independent testing of products

Can we have some independent testing of web search engines?

thih9• 5 years ago

Rtings seems a similar project; some of their test results are free, some are paid. They have a wide set of comparison tools, they test for a lot of features and they document their testing procedures well; it's a really nice change from SEO spam articles.

I recommend their tests of headphones [1], I especially like how they measure breatability [2]. They also have a page about printers [3].

[1] https://www.rtings.com/headphones

[2] https://www.rtings.com/headphones/tests/design/breathability

[3] https://www.rtings.com/printer

ohashi6• 5 years ago

I'm in a strange position here. I fundamentally agree that reviews for the most part are absolute steaming pile of crap of a space for many niches. I work in one of the worst - web hosting reviews. It's plagued by fake affiliate reviews dominating basically every search result. I've been trying for 10 years to run a company that did reviews differently in the web hosting space.

Full disclosure, I have affiliate links on companies that have them too. But I also list companies without them and it has had zero bearing on any result in 10 years. In fact, when I launched I had to beg the CEO of the top rated company to create a special affiliate program for me. Why? Because he didn't believe in review sites and affiliates in the space. It took months, but I told him if he didn't create one, what I was trying to do would never have a chance because I'd never make a dollar - you're the top rated company. I want to do something different, but it needs to remain somewhat financially viable and if you don't have a program I'm dead before it starts.

So what happened in those 10 years?

Honestly, not a whole a lot. I have mediocre rankings (often page 2-5) on some of the most competitive terms on Google. I can't afford to buy the links my competitors do because they make 10x or more what I do pushing the highest paying affiliates and designing for conversion. The site has some traction within niche communities - especially the WordPress hosting space - because I also run annual performance benchmarks (https://wphostingbenchmarks.com) where I document and thoroughly test most of the meaningful players in that space.

It makes a couple grand a month, I've disclosed the revenue publicly on IndieHackers (https://www.indiehackers.com/interview/reviewsignal-e1ddcc26...) and it's gone down since then.

The data I'm providing is almost surely the most transparent data tracking the industry and maybe the least biased (the reviews work by analyzing Twitter sentiment at scale - everything publicly documented in terms of ranking algorithm and published comments).

But outside little bubbles in communities that care, nobody noticed. Google doesn't care. Google happily ranks affiliate sites spending six figures buying links off apache.org and other open source projects (look at those sponsor lists on a lot of open source projects - hosting/gambling is a bad sign).

I got excited when my work fighting against .ORG registry price increases and sale at ICANN (https://reviewsignal.com/blog/2019/06/24/the-case-for-regula...) got a lot of press, even getting cited by the California AG in his letter which effectively killed it. I got backlinks from a lot of large news sites and traffic. I honestly saw no meaningful improvement in rankings or traffic.

So I'm stuck, I keep the sites running - part time - mostly between other projects. I've moved back heavily into consulting and other projects because being an honest affiliate - I can't compete. Providing honest, transparent data and presenting it with the goal of informing versus pushing sales is a terrible business model. The majority of people simply don't care. A lot of 'in-the-know' folks read and get informed by my work. They advise their clients using it, and I never see any financial benefit from it. The broader world, especially Google, doesn't know or care.

Is the root problem affiliate links? It certainly skews incentives and pushes manipulation. If we removed them, what fills the void? Ads? Sponsored content? Something else? I don't think the problem goes away - there is so much money in some of these industries and the stakes are so high. Companies and people will take advantage of it one way or another.

How do we identify honest / good content from the garbage seems to be the bigger question. After 10 years, I'm don't have an answer and I'm certainly not being noticed.

hubraumhugo5• 5 years ago

I was always fascinated by the idea of "field testers" testing products in their daily lives and writing a short review about their experience. A cook would write about a chef knife, a post officer about comfortable all-weather boots, a craftsman about a tool etc.

The biggest problem would be how to incentivize such people, but gamification and some monetary rewards from the community could probably solve this.

frabcus• 5 years ago

Which? https://www.which.co.uk/ is great, based in the UK, not very expensive to join for a month to look up one product.

xxs• 5 years ago

The only reviews I respect nowadays are... tear downs. No innards, no fun. Unfortunately printers would be proper hard to tear down on a review.

Mezzie• 5 years ago

I'm just imagining this rule meaning the only printer you can buy is the one from Office Space. Beating the crap out of a machine counts as tearing it down, right?

Beta-73• 5 years ago

I've also had the same problem. For a while i could get good recommendations/reviews by adding "reddit" to the query as i could find good information there, however i think that the sites have caught up on that and now i only get 1-2 results from reddit.com and then the rest are other sites that reference reddit so they are in the results.

ghostpepper• 5 years ago

At the risk of any SEO-blogspam people reading this and adjusting their tactics, you can filter by domain, eg.

product name review site:reddit.com

ramraj07• 5 years ago

This is far worse in a place like india. It’s pretty much impossible to find anything technical or product related that’s localized. The only non blogspam source often is <vomit/> quora.

For example. Most construction happening in india now uses something called M-Sand. I actually CANNOT find what the hell it is except from some company websites or random YouTubers blabbering non engineering garbage about it.

ummonk2• 5 years ago

What is there to know about it? Looks like it's just sand that's produced by crushing stones in an industrial process, rather than relying on rivers to do it for you. It's probably used as an aggregate for concrete.

Edit: a quick search on Google Scholar brings up papers like http://www.kresttechnology.com/krest-academic-projects/krest...

otherotherchris5• 5 years ago

You've probably already google-fu'ed this, but its ~4mm crushed granite used as a concrete-sand substitute where granite/gneiss sand isn't available.

Most construction materials businesses sell locally by word of mouth, and if they paid for a website at all it's old and still using http. So Google drops them.

889135272• 5 years ago

You can't even search for GitHub issues anymore. You'll get some mirrored site that has the discussion, and from that webpage, you can't even get a direct link back to GitHub.

fxtentacle2• 5 years ago

Yeah, I also noticed that for "tiptoi wiki" Google ranks

https://github-wiki-see.page/m/entropia/tip-toi-reveng/wiki/...

higher than the actual GitHub wiki that all of the content was copied from

https://github.com/entropia/tip-toi-reveng/wiki/Languages

BTW, building my own interactive book was a great thing to do over Christmas.

ljm• 5 years ago

Github issue clones and StackOverflow clones pretty much dominate most of my programming related search results these days.

Unfortunately DDG isn't much better.

It takes a lot of effort to finesse a search query such that I can get a good result, like a link to documentation or a personal website where someone wrote something up (which is often more thoughtful than what Medium and Dev.to offer).

hidden-spyder3• 5 years ago

uBlacklist [0] solves this. The filters in [1] let me block all those clone sites.

[0] https://github.com/iorate/uBlacklist

[1] https://github.com/rjaus/awesome-ublacklist

FranksTV• 5 years ago

Add "site:github.com" to your search and just get results from github.

holoduke• 5 years ago

One thing what helps me getting better results is adding the keyword 'forum'. Usually still have okayish results. Forums are nowadays the only source information not spoiled by SEO etc

aix14• 5 years ago

That's a good tip. I too do this quite a lot; forums can be quite noisy but also a great source of first-hand experiences.

tcoff91• 5 years ago

Google used to have a great discussion search mode that only showed forum results. I miss it dearly.

pxmpxm6• 5 years ago

Beat me to it

megablast2• 5 years ago

Consumers laments the effect of consumers consuming too much stuff.

Maybe buying more useless stuff isn't the solution???

freeqaz2• 5 years ago

In the USA, Consumer Reports exists to help deal with this. It's a yearly fee to subscribe, but it's as unbiased of reviews as you can get!

x0x0• 5 years ago

Right, but the majority of folks on here squeal when consumer reports or wirecutter want to be paid for their recommendations. Buying the devices, comparing them, etc all cost money, and refusing to pay inevitably leads to the state we're in...

I don't have a solution for this; I just think we're in the exact state we would expect.

mthoms• 5 years ago

Pro tip: You can often get free access to Consumer Reports paywalled website with a library membership.

seanp2k25• 5 years ago

Yep; a huge portion of Google results, especially for spicy searches like “best ______ 2021” are just lists of affiliate links to top 10 selling items on Amazon from made-up brand names that are rotated once a product receives a few bad reviews.

It’s really hard to find legit review sites; at least Wirecutter seems to actually test things, but sites like SeriousEats, OutdoorGearLab, Carryology, DCRainMaker, SoundOnSound, Adventure Journal, Magnetic Magazine, The Loam Wolf, etc that are quite niche / domain-specific are where I go for actual trustworthy reviews.

I agree that Google seems to be dominated by clickbait ad-riddled BS SEO sites now more than ever, and I can’t help but think that Google is allowing this to happen because it pays the bills. I’ve posted about this before, but at the end of the day, Google and FB are advertising companies trying to be more than that. The difference to me is that I’m willing to reward actual reviews and effort with rev share if I decide to buy something reviewed, but I’m super unimpressed with all the irrelevant ads we still get in 2021 despite having so much personalization data about users.

Another thing that advertisers don’t seem to understand somehow: if I searched for a thing or even clicked through a FB ad and bought it, the chances that I’m also interested in buying a similar thing in the next few days / weeks are drastically reduced. They seem to be totally missing this signal, showing me ads for some category of thing I already purchased for a very long time after I don’t need any more suggestions.

Lastly, I would literally pay per month for an Amazon search that filters out all the fake brands. If I search for “webcam”, there are half a dozen brands I want to see, yet instead I’m forced to sift through piles of junk that I would never even consider purchasing to find what I’m looking for. I’ve heard that Amazon knows this is a thing but chooses not to fix it due to some psychological allure of sifting through the junk to find the nuggets of gold. In the worst cases, I have to use Google to find stuff on Amazon because their own search is so horrendous, with the categories being an absolute joke.

littlecranky6710• 5 years ago

And just at this moment, we have hordes of affiliate marketers working 24/7 updating their "articles" to say "best _______ 2022".

coliveira2• 5 years ago

They seem to be totally missing this signal

Ok, but how would they know that you already bought the product? As advanced as ads are, they still don't have the Amazon confirmation telling them you already completed the purchase.

travisjungroth2• 5 years ago

the chances that I’m also interested in buying a similar thing in the next few days / weeks are drastically reduced

This is a comment sentiment (I’ve already bought a fridge! Don’t need another!) and it is a bit of a failure mode. But, I think value on advertising around recent purchases to people is super high. A recent purchase, although often wrong, is one of the best signals you can get. So much purchasing happens in clusters (setting up a space, picking up a hobby, etc) that a specific person is in buying mode for a specific topic is crazy valuable. And there’s splash damage on the wrong ads. Maybe you don’t buy a second rice cooker, but the ad reminds you to get a toaster.

enduroman2• 5 years ago

Dcrainmaker has been "pay for play" for at least the past 10 years. He's very thorough, but not altruistic. There was a guy on Slowtwich that shared a conversation with him that was enlightening to that fact.

fault1• 5 years ago

Google and FB are advertising companies trying to be more than that

This also seems to be what Amazon is also devolving into. Amazon ads is the fastest growing part of amazon!

mad1823• 5 years ago

Not surprising. It's incomparably quicker and more profitable to list a few top selling products with affiliate links and a bunch of relevant keywords, than to spend significant time and money actually researching and comparing these products in a meaningful way.

cwilkes• 5 years ago

I never had problems with fake products on Amazon until I looked for a usb thumb drive for the Arlo camera base station.

There’s a dramatic difference in price for 1T sizes. Some at $30. Others at $150. I couldn’t understand it.

One of the 0 star reviews said it was actually a 32Gb drive that somehow fools the OS to think it is bigger. Not sure how that happens but it steered me away from any of the cheaper options as I don’t need a headache.

tomcam• 5 years ago

trying to be more than that

Citation needed ;)

greggman33• 5 years ago

I can’t help but think that Google is allowing this to happen because it pays the bills

Google is not that short sighted. They know that if people stop trusting it to give good results then they'll lose their market.

I suspect the problem is just harder than it seems.

dghughes• 5 years ago

but at the end of the day, Google and FB are advertising companies trying to be more than that.

Looked at Netflix clicked on a movie that I though may be interesting, it was overdubbed in English since it was a Korean movie. I watched maybe 5 minutes of it then exited. Go to YouTube and suddenly Korean videos are being suggested.

At a car dealership with my sister she asked me to be there for support when she leased a car. At home same day hours later YouTube again it is plastered with "how to buy a car" videos. I hadn't been looking for a car, never searched for anything like that in the previous days ever.

KarlKemp4• 5 years ago

Another thing that advertisers don’t seem to understand somehow: if I searched for a thing or even clicked through a FB ad and bought it, the chances that I’m also interested in buying a similar thing in the next few days / weeks are drastically reduced.

This is a ten-year old talking point. Why people continue to insist on this falsehood is beyond me. Do you think these advertisers are stupid?

-> buy refrigerator. gets lost during transport / is damaged / “oh, I wanted the 550T, not 505T” / “works great, let’s also replace the one in the beach house” / etc.

You have to remember that these possibilities are competing with other ads that have extremely low rates of success.

b1122• 5 years ago

If Google really cared, dropping all pages with affiliate links, would get rid of 99.9999% of the junk.

And no, I don't care about the tiny, tiny number of legit reviews with links. I doubt I'd see them anyhow.

Siira• 5 years ago

Isn’t the obvious solution to include the age (or the number of sales) of a product into its rating? This way they can’t just rotate the product/brand names.

bfirsh• 5 years ago

Is there some aggregation of these domain-specific review sites anywhere?

Wirecutter has been pretty hit-and-miss for me recently. I have found my best product recommendations through a bunch of random blogs who have some particular expertise.

prmoustache5• 5 years ago

If you would literally pay for an amazon filter, why don't you put your money where your mouth is and buy from dedicated shops?

Initially because I hate monopoly but also because looking for something decent on amazon is a major PITA I avoid amazon for nearly everything. If I need a computer, I buy direct to the vendor or to a computer shop. If I need new bicycle parts I go to an online bikeshop, if I want a new sampler groovebox, I go to a store selling home studio gears. Well this isn't totally true as I now favor second hand stuff but the reality is Amazon is not very effective at offering you the best things, and not even at the best price. Most online shops nowadays offer shipping time on par with what you get on Amazon.

In then end I don't see any good incentive to use Amazon. Some people will say centralized market place but if searching a decent product takes much more time than finding a good specialized store and getting decent product quickly you haven't really gained anything. Single account for all your shopping? Many shops offer buying without creating an account. Others will allow you to seamlessly create the account at ordering time and password managers make it easy to manage multiple accounts.

jinto36• 5 years ago

SoundOnSound is great, especially since they still host their entire review history. They tend to show up in my top Google results, when searching for older audio equipment at least. For things like mixers and audio interfaces, they've tested quite a few things.

hubraumhugo5• 5 years ago

Same experience here. Researching high-quality products on Google is very frustrating:

- Results 1-3 are ads

- Results 3-5 are blogs that get sponsored

- Results 5-10 are full of astroturfed user reviews

Finding trustworthy and consistent information is so hard that I mostly rely on Reddit for product research. There is a subreddit for literally every product category, and posting a request with your requirements takes much less time than cutting through all the noise on Google.

To make this whole mess a better experience, I'm now working on my own startup that tries to solve these issues.

hw• 5 years ago

In the end search is about relevancy. Trustworthiness of a blog or article is hard to discern by machines. The blogs likely rank high because of relevancy as well as clicks. Sponsored doesnt always mean that the reviews are biased. Reviews on reddit aren’t exactly trustworthy too - there are always going to be competing arguments and reviews for a product, you just have to use your own judgment which plays into your personality, risk tolerance etc.

baddash• 5 years ago

Curious about your startup; how are you planning to solve these issues?

DoreenMichele6• 5 years ago

I am going to remind people that this happens because that's the internet (and world) we designed. If you aren't a coder and you want to earn a living online, blogging is a way to do that and Amazon links are a way to make it pay.

People use AdBlock and don't want to leave tips, support Patreon etc for a blog. Good jobs are hard to find. Telling someone "Get a real job" is often not a viable solution for various reasons.

If you want a better internet, you might stop and think about the fact that it is built by people and people need to eat. De facto expecting slave labor from some people and then designing an internet where those people can hijack your search results to try to eat gets you this.

I know no one actually wants to hear that. But I think that's the actual root cause of this stuff and you won't fix it if you don't address those issues.

I've tried for years to take the high road, to not have ads or Amazon links on my sites, etc. The result is starvation and intractable poverty and everyone telling me to get a real job. (I tried. They didn't hire me and continue to cyber stalk me and steal my ideas.)

Anyway, I know all replies to this comment will boil down to "Quit your bitching. No one gives one damn if you die on the streets of starvation and we are so bored with hearing about your whiny problems." Rest assured, I am not leaving this comment with any expectation whatsoever that it will in any way help me.

But maybe after I die on the streets someone will pause and think "Maybe she had a point. Maybe we designed a shitty system with bad incentives and we are getting exactly the crap we pay people to give us."

Because the "high road" where someone tries to add value and respect the fact that you folks don't want ads or affiliate links etc literally doesn't pay to the point of it will keep you underfed and either homeless or underhoused and then your poverty will be a new excuse to have no respect for your observations that "Hey, people, the system you designed is broken and this is why and how."

no_wizard• 5 years ago

I genuinely agree with you. People need to eat, and live comfortably and be able to afford things like medical care, a place to live safely etc.

I also think the blanket of “all affiliates are bad” is a bit much. I love supporting people who just do a good job. SEO spam is not what I call a good job, it’s a sleazy exploitation of the average consumer.

With that said, I have purchased many things based on what I consider honest good reviews who linked to affiliate programs and I do not regret this. I wish companies (Amazon for instance, as they’re rent biggest right now) simply policed their affiliate programs better to incentive good unbiased reviews sites that Specialize in high quality, and dropped the SEO spammers

nine_k• 5 years ago

Three problems here.

1. Skewed incentives. A company that buys "organic" product placement wants to show it in the best possible light, and increase sales. They don't need an impartial review, so the blogger has a hard time to produce one.

2. Fragility and failures. Affiliate links expire and don't get updated. Ads point at things no longer available. Ads spend an inordinate amount of resources on the viewer's machine. Targeting is inaccurate, despite incessant attempts to track and correlate users' profiles.

3. Direct payment is rarely an option! I personally would greatly appreciate an option to pay $1-2 and read an impartial review of something I'm planning to purchase. Maybe even $5-7 for expensive stuff. But there are very few places that offer this. Those that do try hard to peddle a yearly subscription. Also, it appears that I'm the minority, and the number of visitors willing to pay directly is too low to sustain the authors.

I still hope that it's Patreon and direct support by consumers what the future looks like, not corporate sponsorship and ads.

kelnos4• 5 years ago

If you want a better internet, you might stop and think about the fact that it is built by people and people need to eat. De facto expecting slave labor from some people and then designing an internet where those people can hijack your search results to try to eat gets you this.

The problem is that the sane alternative, where people can get fed while selling quality a-la-carte content directly to the reader, doesn't really exist. If one or more review sites would work to develop a good reputation, I'd be happy to spend a one-shot $3 or $5 on their (for example) wireless earbud reviews. But this thing basically doesn't exist (or at least I can't find it, because of all the aforementioned shit-quality search engine results). Someone upthread mentioned a German site that does this, but then someone replied saying their expertise is limited and their reviews in many product categories aren't that great. Then we have things like Consumer Reports in the US, but I've found the quality of their reviews to have declined over the past decade or so; I've read some of their free content for product categories where I'm already knowledgeable, and I've disagreed heartily with enough of their findings to be skeptical of them.

When it comes to news and opinion pieces, I'd be fine paying on a per-article basis, but we have no established micropayments system, and I'm not paying $10-$20/mo for each of the 50 sites that come up in various news aggregators I read and have paywalls. There are a few sites and YouTube channels that I read/watch nearly daily, so I subscribe to their Patreon or periodically drop money in their donations bin. But the majority of the content I consume comes from various sources, from a list that changes nearly daily.

Maybe we designed a shitty system with bad incentives

Who is the "we" here, though? I would love to change this, but I feel pretty powerless to do so. At least not without making it my life's work, with a very high chance of failing at it regardless.

kragen6• 5 years ago

Well, there are at least two separate, though not unrelated, problems:

1. Good jobs are hard to find, so some people try to make a living instead by influencing what other people buy, in exchange for kickbacks from the sellers.

2. The sellers offering the largest kickbacks are never the sellers offering the best value for money, because if they are, a reseller can buy the products from them, charge a higher price, and spend some of the markup on kickbacks to product shills. This results in a systematic bias toward overpriced junk in heavily advertised products.

3. While some knowledgeable people do still take the time to share their knowledge and unbiased judgments, which is for example mostly how Wikipedia is written, Google and other search engines are increasingly directing search traffic to the product shills instead.

I agree with you that, given problem #1 and problem #3, problem #2 is sort of inevitable, and all three problems tend to mutually reinforce each other. But I think we could reduce either problem #1 or problem #3 by an order of magnitude without significantly reducing the other one. In particular, we might not be able to completely solve the SERP quality problem without solving the jobs problem, but I think we could improve it enormously just by writing a better search engine, which is an easier problem than fixing the entire economy.

I'm sorry to hear you're back on the streets, and I hope your situation improves before you die. I'm glad you're not dead yet because I often find your comments insightful. Happy new year!

littlecranky6710• 5 years ago

I am going to remind people that this happens because that's the internet (and world) we designed.

we didn't design that - they did. I used the Internet in the 90s on my 56k modem and it was mostly ad-free. Stuff was still free, but mostly because people created content out of curiosity and because it was their hobby (yes, I had a website and was using frames and image maps heavily) or university sponsored (IRC, NNTP, Mailinglists). I paid 5$/month for a shell account, later shared the cost for a dedicated server with a couple of friends (we had a traffic limit of 50GB/month) to offer content and services to other people - expecting nothing in return.

Then companies realized how much money was in, and suddenly all good, federated and free services were overrun by spam (mostly NNTP). To this day, companies like FB, Insta, Google etc. try to lure as many content-creators to users their "free" sites, while they actually pay with their data or the data of your content consumers - while the data is used to steal attention to show you ads for stuff you don't need. The amount of GDPR violations by those Megacorps is immense as they very well know that presenting a "reject all tracking" button as demanded by the GDPR ruins their business model (less than 10% actually agree to being tracked when presenting them the choice).

My solution to all this is easy: Ban all advertising on the internet, or, at least advocate to as many people as possible to use adblockers. Only when ads are gone, will people spend money on products such as FB and Youtube, and only then can there be actually competition - because right now, you can't compete with "free" services with any other model than using ads+tracking yourself.

greggman33• 5 years ago

I'm not going to say "quit your whining" but rather, maybe this isn't "the world we built", maybe it's "the way the world is". I know I've been guilty of thinking the later when I should think the former but really don't know what we'd change. Do you really think the internet would have flourished if it was pay per view (or similar solution)?

I remember using online services in the 80s where they charged by the minute. CompuServe, GEnie, and even just my ISP when I first got internet in the 90s. Data of 1 but I know that gave me a "use it as little as possible" mindset.

Further, if some groups set out to charge and some other groups offered "free with ads" I'm pretty confident "free with ads" would win (see Radio, TV, Podcasts as other examples of "free with ads"). So, short of outlawing free (which would likely never happen), I don't see how we'd have not gotten where we are at the moment.

That isn't to say we can't do better now but I'm less confident we could have conscientiously directed ourselves to better. I think natural forces got us her and will take people experiencing better to get them to switch better.

To put it another way, without tasting the "free with ads" kinda sucks, few would be willing to fork over $ for better.

tablespoon3• 5 years ago

People use AdBlock and don't want to leave tips, support Patreon etc for a blog. Good jobs are hard to find. Telling someone "Get a real job" is often not a viable solution for various reasons.
If you want a better internet, you might stop and think about the fact that it is built by people and people need to eat. De facto expecting slave labor from some people and then designing an internet where those people can hijack your search results to try to eat gets you this.

Even if no one used ad blockers and tips were commonplace, we'd still have these problems with crap SEO and affiliate links. It's like with technology products: they say "if you aren't paying for it, you're the product not the customer," but even if you are paying for it you can still be the product (e.g. your TV manufacturer putting ads in your expensive TV's menus).

Ultimately the problem is greed and the ridiculous degree of tolerance of it, unchecked by any cultural expectation of quality or virtue.

iypx• 5 years ago

For my entire life I've tried to become a programmer, went as far as studying CS for three years, never managed to get a diploma as I ran into some financial and health issues... you mention

I know no one actually wants to hear that. But I think that's the actual root cause of this stuff and you won't fix it if you don't address those issues.

I think the real issue, is, the world is much bigger than the first world countries. I would love to have a diploma, to get a "real" job as a programmer for some random company, to add that "value" to the internet, to design useful software for actual people, etc. etc. But I was born in Eastern Europe..

I tried freelancing a few times through gac, rac, and scriptlance which later got "globalized" (all acquired and killed by freelancer.com, yay capitalism).. and I just can't compete with indians on that site, way too many of them, somehow quoting 5% of the price promising double or triple of what I can do somehow...

So I tried looking for that "Real" job, except I was born ugly as hell with broken teeth, so nobody would even take me seriously. Even one of my professors in uni told me once: "See.. if you could do less drugs and study more, I bet you could get an A, it's a B- for now..".. I never even tried drugs, but apparently I'm so ugly half my profs thought I was drugged. As far as I can tell, It's just nepotism right and left, people only hiring other pretty-looking-people-alike for both normal and startup-like jobs where I come from. Zero chance for me.

So on the brink of giving up on everything, I took a construction job for 6 months, which allowed me to save enough money for a plane ticked and my first month of rent in UK. Now I'm a Reach Truck driver in a warehouse.. making >triple the salary I could make as a programmer in my home country, but slowly destroying my back and knees every d day.

Funnily enough, I've met quite a few programmers, lawyers, accountants, pharmacists and even a surgeon through my warehouse jobs in the last two years. Quite smart people, a few broken teeth here and there, a couple disfigured faces, husbands, kids, sick family, plenty stuff...

The internet is just bullshit, just like most of this society. Making money, jobs, starvation, HA! I don't even know what I'm trying anymore or why I still keep reading on this site.

Sorry for the rambling, your message just resonated somehow for me so I felt I should say something. Good luck to you!

yourapostasy10• 5 years ago

After reading your comment, I visited your page and glanced around. Am I right in interpreting your specialty is coming up with how to turn a better phrase (writing)?

If so, then that's a very underappreciated domain at this time (much to our detriment, but that's for a different discussion). Not just in tech, but everywhere. The entire field has been aggressively monetized to near extinction; whatever writing doesn't directly impact revenue/profits, gets dumped onto line workers. And the results show.

The quality of writing and lucid ideas found in advertisements and landing pages is markedly higher than from say, in a somewhat-technical presentation to managers-of-managers. I've sat through more than a few quite underwhelming presentations that violated some of the most basic principles of presenting ("don't mumble hoping the slide will carry the presentation", "don't read the slide word-for-word", etc.).

There is a dearth of assistance for coaches to help improve those kinds of "writing". And a huge number of aspiring, gung-ho FAANG employees who are wondering why management won't listen to them. There's a gap to be filled here, and people with the disposable income to pay for outcomes.

Leverage your non-technical sphere as a strength (I liked your pull request journey, it is true that coders don't normally perceive and accommodate skill stratifications in our docs, a byproduct of the monetization of writing that killed too many technical writing/librarian positions in our industry). Perhaps look into positioning the writing assistance you offer to technical staff as a way to communicate to non-technical managers, where you help translate the technical concepts that have to be broken down to you due to your non-technical background, into ideas the managers readily grasp. It would help if you retained a lot of the corporate-speak that Rands in Repose sometimes writes about that you doubtless saw in your Aflac stint, and knew when to judiciously use it.

Maybe investigate some offer like helping coders write emails to managers. If that gains traction, offer small-medium-large presentation improvement coaching, based upon the length of the presentation (15, 30, 60 minutes). Where shorter lengths increase in cost. Then if that gains traction, offer coaching linked to an annual performance goal of "improve communicating with managers".

bsanr• 5 years ago

If you want a better internet, you might stop and think about the fact that it is built by people and people need to eat. De facto expecting slave labor from some people and then designing an internet where those people can hijack your search results to try to eat gets you this.

People don't/can't pay for things anymore because trillions of dollars have been siphoned out of the holdings of the middle and working classes over the past 50 years. If not for piracy and subscription services, the music and film industries would have collapsed years ago. If you made people pay for the things they used to, they... wouldn't. Because they can't.

prmoustache5• 5 years ago

Don't mistake "people using adblock" with "I don't have a valid business model". Adblock users are a minority among people browsing the internet.

massysett2• 5 years ago

I agree with you. I think Google works very well. People expect to go on Google, pay nothing, and get good information. So industries have arisen that do just that. I find it amazing that I can type obscure questions into Google and get content custom crafted to answer my question.

Often the answers are not the best. What can I expect? I paid nothing for them! When I want good information, I pay for it.

literallycancer2• 5 years ago

Most SEO spam I've seen lately looks like it's produced by neural networks and maybe curated by people afterwards.

Have you tried making some content available only to subscribers?

pers0n2• 5 years ago

Even if you use patreon, people get greedy and eventually will sell out and put up reviews of stuff for affiliate links, free products for reviews, etc.

hemloc_io• 5 years ago

When I do product research I generally search for whatever the enthusiast community lives and usually there are guides there. Blogs are exactly as you describe, either built in ads or useless.

For common items it's kinda difficult, but still possible! (Printers had a pretty active subreddit/community for example.)

nicoffeine• 5 years ago

Buy a Brother Color Laser printer. $250 and the ink never dries. I haven't thought about printing other than pressing "print" in 6 months.

NiceWayToDoIT• 5 years ago

I have Brother printer, full can of paint and sensor saying that is empty, I cannot do anything with it, and that is just one in long list of issues I had with printer...

rejor121• 5 years ago

This is something I have a problem with as well. Search results typically include the first five results being paid ads.

Everything else for the first couple pages are typically review sites that are not really reviews.

YouTube isn’t much better.

I’m trying to addresss this in my own way by making long term product reviews of stuff I own, but I wish other people and companies did the same thing.

Volker_W29• 5 years ago

I’m trying to addresss this in my own way by making long term product reviews of stuff I own, but I wish other people and companies did the same thing.

Link?

ren_engineer• 5 years ago

Seibel doesn't seem to understand how affiliate ads work either, Google doesn't make money from them. Google would be better off just applying a ranking penalty for any page that has numerous Amazon affiliate links. 99% of dedicated affiliate sites are trash

littlecranky6710• 5 years ago

Or at least allow us to block them in the search settings, but Google shows zero interest in doing so. Just as they do not want to have a flag "do not show paywalled content" which is often asked for.

h3mb3• 5 years ago

YouTube's latest brilliant "innovation" to hide the dislike count made this problem so much worse too. When e.g. trying to make a purchase decision, you can't quickly skim through a bunch of search results any longer. Instead you are forced to sit through so much bad content that you could've previously avoided. Maybe that was their real motivation too – to boost some watchtime metrics.

Blackthorn• 5 years ago

Even before that, I hate hate HATE how they don't have a way to say "don't ever display this channel in search results". So much spam.

annadane• 5 years ago

B-b-b-b-b-but I've been told that some people don't look at likes/dislikes and that makes it ok! Oh well, I'm sure some product manager got their raise for removing such a useless feature

ranit• 5 years ago

Especially sad since those affiliate links to Amazon mostly resulted in "This product is currently not available" sites.

This should not be considered "especially sad" in my view. It would be worse if these links worked, thus generating additional revenue for these "web players" (pun intended).

littlecranky6710• 5 years ago

The problem is, they do. Once I clicked the link, all my purchases in the next 24h will be credited to the affiliate. They are incentivised not to remove content about products that are no longer manufactured.

ashtonkem• 5 years ago

This is been my experience with tool reviews as well. Searching for “best intro <power tool>“ yields a bunch of low quality SEO that’s just pulled from Amazon.

I think the interesting question is this: why is this happening? There’s always been a battle between Google and SEO black hats, but I can’t remember the last time it got this bad. Is Google just temporarily losing, or have they lost the will to fix this at all?

Volker_W29• 5 years ago

This is been my experience with tool reviews as well. Searching for “best intro <power tool>“ yields a bunch of low quality SEO that’s just pulled from Amazon.

Cannot reproduce.

yourapostasy10• 5 years ago

I've given up using Google or any search engine for that matter when it comes to tools purchasing decisions (other product categories too, for that matter). First use, I buy from Hazard Fright. If it gets janky on first use or wears out with enough use, I search in the forums where the trades hang out in discussing what they use, wince while reaching into my wallet, and pay for a built-for-business grade tool. There are a few YouTube channels I refer to as well (examples are Project Farm and Mike Festiva) for tools and consumables recommendations.

I always observe what tools trades carry when they work on my properties, and chat with them over the drinks and snacks I offer them, on what they like and don't like. That figures heavily into my decisions.

Google's main purpose these days for me is non-mainstream, far niche searches. I've taken to looking into deploying my own ArchiveBox to just store my own curated search results that I like. When I initially noticed the SEO spam around 20 years ago, I began to prepare for it by manually saving into text files the page links off of search results that I used, and that has helped me a lot. Ironically, I'm building my own personal Yahoo Directory. I don't see any solution on the horizon to SEO pollution getting better, so I want to scale up my solution.

codingdave2• 5 years ago

If you want to solve the problem of product research in an old-school way, Consumer Reports still exists. Their business model for almost a century has been to produce independent reviews of products, and charge for the reviews. It is also run as a non-profit. I'm honestly surprised how little I hear about them. Not that any organization is perfect, but they have been the poster child of success via paid content since the 1930s.

It also makes me think that part of the problem is not only that Google's results are getting worse, it is that much of the population goes to the internet for all problems. Whether it is googling or asking on social media... the "front-line" information on the internet is simply not reliable anymore.

fnord773• 5 years ago

I don't trust consumer reports any longer. they have amazon affiliate links. I suspect the current CEO is selling them out, bit by bit. Also, their current web format makes it hard to find recommendations on used cars.

KB• 5 years ago

Consumer Reports is my first stop whenever I look to purchase some non trivial item. I typically use the top five from their recommended list to quickly narrow down my initial result set. Then I'll cross reference with Wirecutter to see if they are in agreement (usually are). If I'm down to one or two choices at that point I'll try to find some unbiased reviews on YouTube. Not a perfect system, but I find it works pretty well compared to just going right to a Google search result.

jerkstate• 5 years ago

Yeah I sing the praises of Consumer Reports all the time. I think I pay $10 annually for a subscription and every time I buy or recommend something I check with them and so far have not been let down. Their reviews almost always include objective measurements and durability testing, it’s really surprising how people miss them among more modern options.

jimmaswell9• 5 years ago

What more can Google even do about the onslaught of ever-evolving SEO spam besides hardcode some arbitrary "winners", which would have its own set of problems? It seems like a very hard problem.

23B1• 5 years ago

Sorry I have to weigh in here and say that Consumer Reports is a pay-to-play service now, where often the worst products & services are not the best, and often times criminally bad.

As an example, I used them for moving services for a state-to-state move. Turns out the top three or four moving services are merely dispatchers run by a single company, run by a convicted felon out of Florida under a rotating number of businesses and cutouts.

When I contacted Consumer Reports to let them know about the many tens of thousands of complaints about the companies they were ranking highest, they referred me to their attorney.

stanmancan• 5 years ago

I wish CR did -more- reviews in each category. I know it's next to impossible to thoroughly review every single product in the world, but I would pay 10x the subscription fee if I could reliably go there and find all the current models available in the different categories.

andrewf2• 5 years ago

Aw man. I just signed up for the $59/yr magazine + online access. I believe the bulk of what you're saying, and that CR's methodology has value. That said the following was also part of my experience on consumerreports.com -

In the bottom footer, I click on "Ad Choices". I'm presented with a list of advertisers in a TrustArc-branded dialog. To opt out of being retargeted by consumerreports.com, there are checkboxes for three vendors: Microsoft, LiveRamp Inc. and Google Advertising Products. For seven other vendors, there's no checkbox, just instructions to visit the website: Amazon Advertising, Bidtellect Inc, Comscore B.V, Facebook, Google Inc, Kibo Commerce, and Twitter.

Also in the footer (maybe only for California residents such as myself?) there is a "Do Not Sell My Personal Information" link. It opens a OneTrust-branded dialog with the option to disable "Share My Information with Third Parties on Digital". It also declares that "If you are a Print or All-Access Member and receive Consumer Reports magazine or Consumer Reports on Health through the mail, we may share your name and mailing address for direct mail purposes with selected companies offering products or services that we believe will be of interest to you." I followed a link to a separate page, which required me to copy-paste in my just-received membership number, to opt out of this.

evilduck6• 5 years ago

CR today: "After extensive review, LG Microwave model AX7J3498-2 is best avoided"

LG tomorrow: "Announcing LG Microwave model G8A867FL2-B! Also, model AX7J3498-2 will no longer be available"

Consumers two days from now: "Well, there are no bad reviews for model G8A867FL2-B when I look it up, I guess it's ok to buy"

I like the idea of Consumer Reports or in depth reviewing but manufacturers have learned to game it too.

tmaly• 5 years ago

My friend in Germany pointed me to a site like CR but with a better track record. I have to ping him for the name again.

lr4444lr2• 5 years ago

Seconding this. As recently perhaps as 7 or 8 years ago, I used to shake my head sympathetically at how antiquated their business model seemed, and wondered when they'd finally close up shop. Fast forward to today, with bots, review bribery, astroturfed "buying guides", and paid content even at quality outlets like NYTimes Wirecutter, CR is now my most trusted source on purchasing anything over $100, or a product that could be crucial to my health.

gorbachev• 5 years ago

I've got two problems with Consumer Reports reviews.

1. They aren't often comprehensive enough in their product lineup to be valuable. Obviously a hard problem to solve on product categories like consumer electronics where the product choices can count in the hundreds.

2. They aren't often including the latest and greatest in their reviews either, so I'm quite often not confident I'm getting the best bang for my buck going with their reviews.

lenkite• 5 years ago

This thread has now successfully been diverted from a discussion on Google search results to a discussion on CR.

JumpCrisscross• 5 years ago

The New York Times’ Wirecutter is also quite good.

taurath2• 5 years ago

For now I've been using the NYT wirecutter folks. They do a good job posting their methodology and you can usually dig much further for what problems they found.

DrScump• 5 years ago

CR has biases, and they leak out in various ways. When I was doing detailed car-shopping in the 1990s, I noticed that drivetrains in American-badged cars would show significant problems while Japanese-badged cars with the same drivetrains from the same supplier would be ranked consistently higher (e.g. Isuzu vs Chevy light trucks, Mazda vs Ford sedans).

thelittleone• 5 years ago

I've also noticed something similar when researching mobile phones. Device pages that are clearly automatically created with poor and often nonsensical sentences and repition created by some script. And ranking top of Google. Similarly on YouTube videos that are collection of still images with robot voice reading clearly copy texts with almost zero original content. Again ranking near the top for a topic.

fnord773• 5 years ago

The web has become a crappy place to research products

I'd say most people's interface to the web has become crappy.

mcv4• 5 years ago

I'd say the web has just become a crappy place. It was better when everything didn't have to make money, but was just done for the love of it.

pxmpxm6• 5 years ago

I find myself appending the word "forum" to a large percentage of my google queries these days, to avoid the dynamic you mentioned. This essentially filters the results for real/original content of other people that were looking for the same information as me.

rtpg• 5 years ago

Sorry but what do you expect apart from blogs and review sites? Like what’s the magical result here?

I feel like the fallacy here is assuming that the problem is that Google isn’t finding the good websites. There’s also a simple explanation that the content simply doesn’t exist.

temp8964• 5 years ago

Also because YouTube removed downvotes, it becomes basically useless for searching product reviews.

windexh8er3• 5 years ago

Repeated my search on Youtube to find reviews or unboxing. Most video search results were basically "Youtube SEO" again - the most viewed/top-ranked videos did never show a single actual print run or even the printer available. It was mostly marketing websites turned into video (slowly scrolling/moving over product description or pictures clearly taken from the web). And of course, affiliate links in the description.

This. The web would be a much better place if Netizens didn't continue to monetize garbage like MKBHD, Unbox Therapy and others that provide absolutely no value in "reviews". They're all just shills for product placement at this point. And yet these paid for commercials rack up millions of views and get pushed to the top of YouTube results whether they're of value or not. YouTube doesn't want new creators - instead they've invested in a few that push the content they want based on the advertising dollars that are rolling in. Google isn't search comparative to the early days of the web. Now it's garbage results driven by SEO hackers. A world without Google would be a much better place IMO.

asddubs2• 5 years ago

yup, I was compiling a wish list and searched for a list of good programming books and ran into the same issue. any website that promises a "list" like "top 10 programming books" is usually affiliate spam, with poor recommendations and grammatical errors

schleck8• 5 years ago

Affiliate marketing reviewers are the internet's parasites

Spooky23• 5 years ago

I don’t blame google for this. The web is dead. Facebook and Twitter and Discord took over individual passion publishing, to the point that interesting people are forced to do hideous shit like post 30 post Twitter threads to make a point.

Even in the tech space, while exceptions like Stackoverflow targeted google as an UI, lots of content is buried in unindexed GitHub issues, etc.

kevin_thibedeau2• 5 years ago

Github issues are cloned now by SEO sites.

BikiniPrince2• 5 years ago

Try a few alternative search engines. Google has other priorities these days.

2muchcoffeeman• 5 years ago

For a couple of items I turned to Reddit for some guidance to narrow things down ...

costcofries2• 5 years ago

I always just append the word "Reddit" to any review type searches, it's been tremendously helpful. The peer reviews on Reddit are top notch, give it a go and I bet you'll be pleased.

sjs7007• 5 years ago

I'm just amazed by how many people do this. I always do this too since it seems hard to find quality content just by searching on Google. I try to search HN too sometimes like this but the crowd and topics here a lot more narrow.

sahaj• 5 years ago

We need a new Yahoo Directory.

d--b• 5 years ago

I have been consistently adding -amazon to any product search on google lately. These “blogs” are so annoying…

NetOpWibby• 5 years ago

You used Google Search and then used Google (Video) Search. The issue is an advertising company being the Internet’s default for information.

cokeandpepsi• 5 years ago

I wouldn't say the web is a crappy place just because of Google search kind of stinks and affiliate marketing ( which has been around forever anyway )

teh_klev• 5 years ago

Most video search results were basically "Youtube SEO" again

And the removal of the "Dislike" count so that you too can avoid non-informative content i.e. unbox spam, just furthers how shit this situation is getting.

1vuio0pswjnm7• 5 years ago

Giving Google more data about yourself will help. Improve relevance and Google revenues.

Online advertising will make Google better and that will make you and the world better, too. Therefore, always support online advertising in every way you can.

Look for the helpful crafting printer ads (after you purchase a crafting printer).

In case the reader is unsure, this is sarcasm.

autokad• 5 years ago

Do a google image search for a strawberry. See how long it takes you to find actual pictures of strawberries that arn't from ecom sites and/or stock photos. I ran into this problem because I wanted to train an AI to recognize a strawberry.

Also note, probably none of these are actual strawberries. they are probably fake plastic representations for sales

hidden-spyder3• 5 years ago

I did this. Results 1 & 4 were from Wikipedia and Encyclopedia Britannica respectively. Doesn't seem that bad to me.

clarkmoody• 5 years ago

I've found that searching for a subreddit related to whatever hobby is a fine way to go. People are showing off their creations & results and chatting about recommendations for equipment & techniques.

The "front door" of the Web has become the gift shop. It's time to find another way in :-)

systematical• 5 years ago

When I am researching something I often use this filter:

site:reddit.com + "search phrase"

At least if I want to hear from "real" people.

noobermin3• 5 years ago

until reddit starts getting native content too. We already know people game that site as well.

Also reddit can unfortunately be unsufferable.

Helmut10001• 5 years ago

Google could modify their ranking to include these biases:

- affiliate links? Down ranked

- affiliate links to Amazon? Down ranked further

- page contains advertising? Down ranked

- page contains extensive javascript? Down ranked

- page contains links to Social Media? Down ranked

- post _is_ on Social Media? Down ranked further

- page contains duplicate text, found elsewhere? Down ranked

- page contains many links? Down ranked

- page contains grammatical errors or bad sentence structure? Down ranked

But why would they do it, their whole business is built on these things to exist.

Let's look at some things for upranking:

- page uses a static site generator, e.g. jekyll, mkdocs, hugo? Upranked

- page has a balanced ratio of pictures/text and the text actually refers to picture (+ pictures are not stock imagery): Up ranked

- ... (you continue)

jerrre• 5 years ago

They could, but why would they?

Perhaps you mean: a new incumbent could

tdsamardzhiev• 5 years ago

I'd pay for a search engine where I get to customize this kind of stuff.

orthoxerox• 5 years ago

That's true. At least YT is still better than generic WWW search when you are looking for some specific model or some DIY technique. It's still harder to make a SEO-optimized video than a webpage.

whiskyant• 5 years ago

I almost always start my product searches by googling the term and then adding "reddit" nowadays.

8bitsrule• 5 years ago

This may help: [https://millionshort.com/search?keywords=crafting_printers&r...]

psyc3• 5 years ago

I began to notice some time ago that Google basically disregards my query, and fixates on the lowest common denominator. So, recently I was trying to search for a particular event or quote or something related to some famous person. But no matter how I worded the query, Google ignored everything but the person's name, and returned only fluffy flattering results about the person from popular magazine sites.

So I tried Bing, and the thing I was looking for was result #1. Like how it used to be with Google. So I switched to Bing.

Now after a few weeks of that, I see that Bing does the exact same thing much of the time. Totally different queries + same general subject = identical top ten results.

So. Anyone fancy creating the 2022 version of what Google was in 1998?

marginalia_nu5• 5 years ago

So. Anyone fancy creating the 2022 version of what Google was in 1998?

I've tried, not quite there yet, but it's got its moments.

https://search.marginalia.nu/

hulitu• 5 years ago

Thank you. Looks sane. I'll give it a try in the next days ( a quick search for gopher is better than i expected).

nikanj3• 5 years ago

I’ve noticed same. Google ”amusement parks italy”, get a list of world-famous parks (such as Central Park NY).

vmception• 5 years ago

the searches results aren't the same, you are fingerprinted

its been this way for years now, at least in how it drastically alters search results for different users in the same country speaking the same language

CactusOnFire2• 5 years ago

Not quite as big of an offender, but when I was looking up "Nice Restaurants" with google maps fixed on my city, it instead started looking up Restaurants in Nice, France.

zoomablemind• 5 years ago

Current location bias?

There used to be a way to turn of location priority in advanced search.

crucialfelix• 5 years ago

For me every single result was quality articles or Wikipedia listing amusement parks in Italy.

I'm in Paris

I don't click on trash sites ever. Not sure what else might bias your results.

Try creating a new chrome profile and searching.

iso1210• 5 years ago

ddg gives me:

Amusement parks in Italy, top 5 fun parks you have to visit

20 of Italy's best amusement parks - TravelMag

THE 10 BEST Water & Amusement Parks in Italy - Tripadvisor

10 Best Theme Parks in Italy - Find the best Amusement ...

Family amusement in Italy: 15 excellent parks - Italy ...

Gardaland | The biggest amusement park in Italy

Amusement Park Emilia Romagna Italy | Mirabilandia

The 5 Best Theme Parks in Italy: Italy Logue

THE 10 BEST Water & Amusement Parks in Tuscany - Tripadvisor

Category:Amusement parks in Italy - Wikipedia

Not great, but seems reasonable. Google gives similar answers to be fair.

pkulak• 5 years ago

Huh. My first result is pretty darn helpful and relevant:

https://public.kulak.us/google-search.png

mleonhard• 5 years ago

I get extremely relevant results from Google for "amusement parks italy". I use a privacy proxy (VPN) and a browser in privacy mode. Perhaps Google only switches into guess-what-you-meant mode when it can link your search to their profile of you?

Volker_W29• 5 years ago

I only got sites about amusement parks in italy.

throw_m239339• 5 years ago

I began to notice some time ago that Google basically disregards my query, and fixates on the lowest common denominator.

Which is funny. Google beat other search engines in the early 2000 because it actually did find what people where looking for, not what the "search engine" wanted people to see.

Now it's more and more the latter, I imagine because it's more lucrative for Google to display the results advertisers pay for...

That's really the product of the lack of competition in the search space. Nothing more. Why should Google bother? It would take billions in VC for a competitor to truly threaten Google dominance on search.

Same with Youtube. Youtube straight out doesn't care anymore about search terms and will just show some results Youtube "cooked up" for the user, unbelievable...

noizejoy4• 5 years ago

it actually did find what people where looking for

I believe that is more a function of the web being less commercially relevant back then.

And every small and then massively growing online community goes through the same evolution:

While the community is small, commercial value is small enough, information is less tainted - and once it becomes big enough, the commercial value of that community becomes worthwhile to game.

anigbrowl• 5 years ago

Consider also the possibility that they do want to deliver good results but their algorithms just passed their useful limit some time ago and people can game the system faster than they can improve the system, but there's too much money on the table to ever admit this in public.

hk__23• 5 years ago

So. Anyone fancy creating the 2022 version of what Google was in 1998?

I don’t know if it works well, but there’s Neeva [1]. It started as a search engine with a paid subscription model but then switched to a freemium pricing with a premium tier that will come "soon".

[1] https://neeva.com/

LeoPanthera• 5 years ago

The problem is that Google was only good in 1998 precisely because it was pre-Google. Now the web is SEO'd to hell and actively trying to prevent you from getting good search results.

A "new old Google" would only be good at searching the 1998 web, and if all you want is nostalgia, http://theoldnet.com is right there.

tomrod3• 5 years ago

So we need a new web.

Fricken2• 5 years ago

My experience is that Google continually improved up until around 2014 or so. For the last 3 or 4 years it has slowly been getting worse.

chiph• 5 years ago

Agree. The web material being searched is bad at the source. So there's little that a search engine can do to improve it. As the adage says - garbage in, garbage out.

mad1823• 5 years ago

Exactly.

I don't think its only the google search algorithm to blame, this has a lot to do with the extinction of old school forums and blogs. These days large part of the real discussions and posts without financial motivation have moved to facebook, whatsapp groups, discord, slack and other places behind logins and paywalls which google can't index. What's left in public are mostly blogs and websites motivated by affiliate or ad revenue and SEO'ed to death. So there simply is much higher garbage to valuable content ratio in the public, indexable web.

shadowgovt• 5 years ago

The web was SEO'd to hell in '98 also, for other search engines. Google came along with a better algorithm for sussing out the signal if what content people found useful from the noise of attempts to trick the browser into increasing relevance signal.

It's not entirely clear what the next iteration of algorithm should be... SEO has gotten very good at its game.

more_corn• 5 years ago

Google did good work fighting against SEO over optimization for a long time. Then they gave up and it all went to hell. I stopped using them a few years ago. I found their practice of dropping search terms infuriating. I switched to DuckDuckGo which is arguably lower quality but less infuriating.

889135272• 5 years ago

Even lowest common denominator has gone too far. Querying "Barcelona" gives you 100% search results for the League team above-the-fold. You must search "Barcelona, Spain" to get information on the city, which then gets you direct links to Google Maps, etc.

kristopolous2• 5 years ago

I also get these results and I've never spectated a sports game in my life. These aren't just customized results for different people - if Google keeps a "completely uninterested" personal score I'd have the maximum value for sports

greatpostman• 5 years ago

Yeah their entity resolution algorithms are really annoying. Half my searches come up with some random movie on IMDb

Volker_W29• 5 years ago

If I search for "Barcelona", I get result about the city on positions 6, 14, 21

If I search for "Barcelona city" do not get any results about the Soccer team.

hwers2• 5 years ago

I almost feel as if they have like 100k actual pages that they present (and have looked through manually) and if it's not in that group they just show you the closest one (or say "no results").

psyc3• 5 years ago

I have no knowledge or evidence, but this has indeed become my mental model for whatever Google actually does these days. I do wonder what they actually do. All those engineers, for 20 years. Surely they haven't just been scaling up BackRub! But I find it very hard to believe that they're crawling the whole web. I find it very hard to believe that PageRank is still the same PageRank that we understood in 1998. And it looks like they're managing to editorialize quite heavily, even if they're doing it via algorithm somehow. But again, I can't really discern what they're doing anymore.

So for now, I have to disagree with the "garbage in, garbage out" theory. I don't believe Google has the same goal now that they did then.

JasonFruit2• 5 years ago

Do they say "no results" anymore? It seems like Google ignores parts of your query until you get results, no matter whether they are specifically connected to your search.

momenti• 5 years ago

I suspect the change we are seeing is that Google is now using a neural network to (re-)interpret the search query.[1] Presumably that neural network also calculates some sort of neural hash for document/paragraph/sentence similarity. The upside is that it can correct more typos, intelligently drop irrelevant terms and understand the meaning of the query to some extent. But the downsite is less precision when you know exactly what you want. It sometimes even seems to ignore the quotes syntax for exact string matching. Very frustrating and very poor quality control.

[1] I bet they train these models based on unsuccessful queries as inputs (in which the user did not click any search result), and then the final search query after which the user left as desired output.

otherotherchris5• 5 years ago

From personal experience the final search query is when the user realizes that none of the results contain useful information, or your actual keywords, and rage quits.

The user then asks someone in the office or goes and spends an hour at a university library.

ffhhj2• 5 years ago

Anyone fancy creating the 2022 version of what Google was in 1998?

I'm creating a faster search engine for coders, using good old literal search, with synonyms of programing operations in different languages, ie. array.push in javascript, array.append in python, array[] in php, and so on. The database is loaded in memory instead of huge analytics libraries, and searching is performed instantly. I see no need to protect my DB, since it contains basic snippets, and that allows these fasts queries. I move between several languages and needed a super quick reference without all the SO clones and spam.

Volker_W29• 5 years ago

Link to the Repo please?

groone• 5 years ago

I remember around 2010-2012 it felt really great. You could learn the keyword-fu skill and with few keyword change / reorder iterations could explore topics and find obscure things on the internet. Now that method does not work at all. Always the same results and cannot find specific things. Around that time they started adding ML/AI and now searching with keywords is extremely unsatisfying.

sb057• 5 years ago

I was attempting to find some historical data on the Soviet Ruble the other day, but found the task nearly impossible because Google "helpfully" considers "Russian" and "Soviet" to be synonyms, and so all the results were about the modern day Russian Ruble. I can't think of any other examples off the top of my head, but this isn't the first time I've run into this sort of problem.

Digory• 5 years ago

That’s probably not a bad substitution for the average person, but a bad one for a precise vocabulary.

I get the feeling though, that the need to compensate for mobile misspelling means a dumbing down of precise vocabulary.

aix14• 5 years ago

That's very odd. I just typed "Soviet Ruble" into Google and every single result on the first page is about the Soviet, not modern Russian, currency[*]. This includes the box of images on the right-hand side, all of which depict USSR money.

Not really sure what to make of this.

[*] With the arguable exception of an Encyclopedia Britannica article on "ruble", which covers USSR, Soviet Union and Belarus currencies at the same time: ruble, the monetary unit of Russia (and the former Soviet Union) and Belarus (spelled rubel)...

colechristensen3• 5 years ago

Would you be willing to pay a subscription fee to a search engine?

Karrot_Kream2• 5 years ago

Blogspam and SEO folks are essentially adversaries to good ranking strategies on search engines. In 1998 there weren't many adversaries and it was mostly a technical issue. Now the game is much more complicated.

ColinHayhurst4• 5 years ago

We are trying - with no-tracking principles and practices https://blog.mojeek.com/2021/05/no-tracking-search-how-does-...

ckastner• 5 years ago

began to notice some time ago that Google basically disregards my query, and fixates on the lowest common denominator.

Same here. I was perplexed to see undisputed leader of search engines (which nobody managed to successfully rival, no matter how many billions they threw at the problem) decline in this way.

But now I wonder: is fixating on the lowest common denominator perhaps ultimately the more profitable approach?

Compare this to Amazon. A decade ago, buying on Amazon used to be a fantastic experience; no other retailer could rival it. Now the experience is utter crap, as are many of the products. But that crap still outsells everyone else by a large margin.

Perhaps we are seeing a general shift towards a focus on volume, rather than quality.

rich_sasha• 5 years ago

I think this is how giants fall. The various titans of times long gone - steel, chemical companies, mines etc. were one day so mighty that it was impossible to imagine them faltering. And then they stumbled, then tripped, then eventually fell.

Google, Facebook, Amazon will probably all eventually be replaced by a plucky, energetic and hungry competitor out of nowhere, just as they exploded in the faces of the predecessors.

Agentlien• 5 years ago

I've noticed exactly this. One thing I do from time to time is try to find a song after just remembering partial lyrics. It used to work so well it felt like magic.

The last two weeks I've had two occurrences where the lyrics I remembered contained a common word which was also a brand name. It focused on that and completely ignored the rest of my query. So in both cases I added "lyrics" and now it ignored all but one word which happened to be the title of another song, no matter how I massaged the query.

postingawayonhn• 5 years ago

I've noticed this a lot as well recently. It's not as really the search results themselves that are bad but rather Google simply ignoring key words/phrases in my query.

com2kid2• 5 years ago

Now after a few weeks of that, I see that Bing does the exact same thing much of the time. Totally different queries + same general subject = identical top ten results.

When I worked at Microsoft, the Bing team had an internal version employees could use where you could report if Bing's top results didn't equal Google's top results.

This was nearing a decade ago, so don't read much into it for what they do now.

quitit• 5 years ago

I have no inside information on this, but it's based on my usage of Google recently. I believe Google seem to be guessing at people's intended search query rather than performing the query based on the actual terms.

This is probably great for most people as Google's own data shows that most people do indeed search the same things at the same time, so guessing intention (especially with relation to current affairs and the queries of others) is probably a winning strategy for giving most people what they want - even if their search terms were a bit junk.

The down side is that the ability to hone results by tweaking or rearranging ones search terms goes largely ignored. Previously one could peel away layers of results with such meddling, now it seems there will usually be some word or name in the search query which Google will be affectionate towards, and the results are unmovable from that.

version_five• 5 years ago

Google seem to be guessing at people's intended search query rather than performing the query based on the actual terms.

I've read that google uses "Machine Learning" for their search results which I interpret to be exactly as you say they provide a stereotyped result based on what they think you are searching for (possibly optimized either for what is inevitably clicked on or for ad revenue), instead of actually matching terms.

What this means is that search results may be more accurate in some statistical way, like more people click on the top result, but it also pumps up the number of edge cases where it guesses wrong, while simultaneously making it impossible to tell where the results went wrong because you can't understand how they were generated (compared to eg keywors search where good or bad, the reason you got a result is obvious)

wenbin• 5 years ago

Digital products could have a “finished” state, which is great for users, but bad for companies.

Dropbox could’ve been a finished product in 2012. Simple and focused personal storage solution. But it can’t justify the valuation of Dropbox, the company.

Same for Evernote.

However, could Google be a "finished" product at some point (e.g., 2000)? Probably not. When google was incorporated in 1998, they indexed only 25 million web pages. As the number of web pages grows exponentially, Google as a product needs to evolve, e.g., doing a better job to fight web spams / blackhat seos... The problem is that the web evolves way faster than Google could improve their search result relevance.

olivierduval• 5 years ago

So. Anyone fancy creating the 2022 version of what Google was in 1998?

Well... it's not fair!!! In 1998, most of the content was published by enthousiasts for fun, the "business web" wasn't really existing yet, and Google was a minor player, so nobody was interested in "gaming" the search engine because there was not really some money to take

Fast forward: now, the web (and apps) is a major driver for ANY business, so there's a lot to win (or lose) to game the system. So the SEO and other user-hostile strategies are widespread (and no IA or algorithm will be good enough to change that)

Solutions that could help ? Maybe:

1) meta search engine: using cross verification and ranking with different search engine might help a little bit (more different algo to game for the SEO)

2) user-validated search: allow user to rank website AFTER they checked the page to exclude bad actors. It's gamable too (like comments and Amazon rank)

3) not-for-profit web: excluding business from search engine (but informations without sell is a kind of pre-sale however)

In any way, I don't see a lot of ways to decide an un-gamable system

heavyset_go• 5 years ago

This is awful when you have an error message where the text doesn't vary much, if at all, and you have a varying error code. You'll get results for all of the errors with similar error messages, but with different codes.

bin_bash2• 5 years ago

What are people's impressions of Neeva? I just registered and a few searches seem equivalent to Google but it's a lot faster and cleaner.

sefrost2• 5 years ago

I'm in Canada, signed up and it said it's not available in my region.

yborg• 5 years ago

The whole point is to not be equivalent to Google since Google is an ad revenue engine now, not a search engine.

bin_bash2• 5 years ago

I’m telling you my experience not their goals. The searches seemed nearly exactly the same. I didn’t see any sites that didn’t appear on the other in first glance.

techload• 5 years ago

Not available in my region, Brazil.

adsharma4• 5 years ago

Try "testing" as a sample query. I only see results about covid testing.

Would love to see dictionary like factual results instead of reflecting the controversial topics of the day based on what people are clicking on.

If in fact the idea is that a search engine needs to reflect the political opinions of its users, the incumbents are doing a very poor job.

PaulHoule5• 5 years ago

It is a fundamental problem in search.

One important metric is precision for the first result or "P@1" which was something Google originally excelled at.

I was working at a search vendor where I had the chance to debrief high-level defectors from Google and Bing and found that they (like us, with our patent search) struggled to get P@1 above 70%. They saw personalization as a way forward but also told us that personalization at both of those vendors was limited to a few cheap tricks and their were structural reasons why neither vendor would do it in a deep way.

The trouble is that you can at best make an informed guess about what somebody searching for "testing" wants. You ultimately have to get them into a dialogue which might be "I gotta search for 'nondestructive testing'" or could be based on some diversity ranking so that 'covid testing', 'software testing', 'psychological testing', etc. are represented in the top few results.

adsharma4• 5 years ago

Right - this was called result diversity. But in markets like the US, where avg revenue per user is high, polarization is also high and there is some belief that big software companies have a role in changing societal wrongthink, you get these types of results. For this type of a query, I'd think P@1 would be 0% for some 30+% of the society.

Perhaps that'll be readjusted in a year, when covid is not top of mind for a majority of the users.

Volker_W29• 5 years ago

Can't confirm. My results: https://en.wikipedia.org/wiki/Software_testing https://www.cdc.gov/coronavirus/2019-ncov/symptoms-testing/t... https://www.testing.de/ https://www.bundesgesundheitsministerium.de/en/coronavirus/i... https://www.atlassian.com/continuous-delivery/software-testi...

adsharma4• 5 years ago

Here's the logged out search result in the US:

https://smallpdf.com/result#r=65aec01a358fb33d67f37736c8c099...

tayo422• 5 years ago

Ive wanted to share this similar sentiment but I had no idea where! Google search results are terrible lately. I'm frustrated by it. Im not sure how I feel about his categories though.

I have mixed opinions on recipes. The recipes that do show up are generally not great and seem to be there because of seo. And seo is really ruining the recipes pages them selves (extra content, misleading cook times) At the same time I don't really expect google searches to be a curator of good recipes. There is to much taste involved I think. But reputable site like serious eats almost never show up in results unless I search for it.

Health is a big topic, but health has been full of snake oils salesmens forever too, so its not surprising. You should probably talk with a doctor in most cases anyway?

Going on google for anything related to illegal drug use is a pita, it just brings you too like addiction sites and other useless info ime.

I also don't think that people are creating websites anymore? Where is there supposed to be info for the search engine to crawl? All real user content is on reddit, instagram, twitter and youtube. Maybe medium sometimes. All of that is easier for people to set up but not really set up for search engine to include as a good result. Instagram and twitter are especially bad, black holes of information.

For better or for worse, I usually use google to search to search reddit, then i can at least get some better starting point and return back to google with better terms. I don't really like this because reddit is a bit of a echo chamber.

Ive also stopped trusting searches in fields Im not familiar with because I know how bad the results are in fields I am familiar with. But i don't really know where else to go.

tome2• 5 years ago

Ive also stopped trusting searches in fields Im not familiar with because I know how bad the results are in fields I am familiar with.

The Gell-Mann amnesia of the 2020s.

https://en.wikipedia.org/wiki/Michael_Crichton#GellMannAmnes...

Nextgrid19• 5 years ago

I don't really expect google searches to be a curator of good recipes

I don't expect Google (or any automated service) to be able to rank recipes from a culinary point of view. I absolutely expect Google to be able to detect someone's SEO life story bullshit at the top of a recipe and penalize that for ranking purposes.

And if detecting SEO bullshit life stories is hard, the presence of ads or other marketing-style element is usually a good proxy for weeding out shitty commercial content.

Health is a big topic, but health has been full of snake oils salesmens forever too, so its not surprising

Most countries have national health services that have no (or at the very least less) bias to sell you something. Given the amount of these is finite, a list of their domains can be manually maintained to boost their results and outrank the other garbage.

I know how bad the results are in fields I am familiar with. But i don't really know where else to go.

Same here unfortunately and I have yet to find a solution.

luma• 5 years ago

I absolutely expect Google to be able to detect someone's SEO life story bullshit at the top of a recipe and penalize that for ranking purposes.

Agreed, but those life stories are there because Google penalized the site if they weren't.

A whole lot of this is a problem of Google's own creation along with people adapting their behavior to fit how Google works.

6510• 5 years ago

recipes is a fun example.

The obvious success formula IMHO would be to have a file format for recipes (with a wrapper file format) and display those as search results under a "recipes search" tab. (The user picks his own tabs)

Then, like news results, the results have to be parsed and combined so that there is one main trend for mashed potatoes & gravy with creative alternatives presented in a beautiful crafted UI for the specific purpose that is cooking. One then elects to add the raisins and the cheese and stores this new combination some place private or public under a user name (possibly with pictures, the beef foo bar, soup and desert served with it) for future reference. If the raisins are a wonderful or terrible idea the rating can be merged into the mashed potato search result.

There is no room for good enough in my kitchen!

fancy_pantser• 5 years ago

And if detecting SEO bullshit life stories is hard...

I took the opposite approach in my browser extension, detecting if a recipe is on the page and cloning it to the top of the page. I suppose you could add an accumulator to see how much visible text is on the page outside the recipe on pages where recipes are present to see the distribution of signal:noise. I'd be very interested in the results of a large-scale survey like that!

https://chrome.google.com/webstore/detail/recipe-filter/ahlc...

epolanski• 5 years ago

Where is there supposed to be info for the search engine to crawl?

You actually make me realize how rarely I end up on websites I haven't seen, and how different it is compared to two decades rego.

ssl2322• 5 years ago

Shout out to Plain Old Recipe [1] when it comes to getting rid of the filler/keywords on recipes.

[1] https://plainoldrecipe.com/

sparkling3• 5 years ago

Honorable mention also: https://based.cooking/

mcv4• 5 years ago

Google search results are terrible lately.

Just lately? I've had the feeling Google search results have been terrible for years now.

Someone should really build a better search engine, and preferably one that gives the user more control.

Although frankly it's baffling that Google hasn't implemented some sort of up/downvoting on search results. That sounds like exactly the sort of thing they would love.

srcreigh2• 5 years ago

Re; recipes, look for the "Jump to recipe" button. It's always there, you don't have to scroll.

tayo422• 5 years ago

Thats not really my issue with them. its site like all recipes is always in the top, or the same few websites always dominate results and that low quality recipes are in the top because they seem to game the system by doing that. the 1st result for me for pad thai recipes is a recipe that includes bell peppers and peanut butter.

nullc2• 5 years ago

I'm not sure the recipes are entirely google's fault. There are jerks which are mechanically generating infinity recipes and polluting the internet with them. Resulting in non-viable recipes that no one has ever made and cannot possibly work. It's not even a matter of taste, the results are mostly just nonsense.

silisili• 5 years ago

I would advise against Googling anything medical. Outside of terrible SEO results, WebMD telling you you are going to die, the ads linger around forever.

I remember Googling something I had read about or saw on TV out of curiosity. I got ads about help for said disease I don't have for at least a month...

dleslie2• 5 years ago

It's funny, I've gone through a number of search engines. Webcrawler, altavista, lycos, yahoo, google and now duck duck go. DDG is the only one that hasn't been an abrupt switch. I would search DDG, then !g if I didn't find a tolerable result. But that's increasingly changed in the last few years, to the point that I have stopped using !g altogether.

More often I want !w, !gh, !mdn, !msdn, !v, !osm or similar.

It's that which makes DDG great: instead of being one search engine to fit all, it's a portal to services with specialized information. If you're using DDG's basic search to find specific information then you're using it wrong.

TOGoS• 5 years ago

Aw neat; I didn't know about that feature of DuckDuckGo. I used to have similar keywords configured in my browser ("gg", "wiki", etc), but as the years go on I've become less and less willing to invest that time every time I switch to a new computer. Now I have a reason to set DDG as my default search engine again.

mgh22• 5 years ago

DDG is built on Bing, and their business model is the same as Google's. As long as you share this last part, search content quality will suffer.

We will need a company that does not rely on this model- ex: Apple

solarkraft2• 5 years ago

But that's increasingly changed in the last few years, to the point that I have stopped using !g altogether.

Funny thing. I'm used to DuckDuckGo results being unsatisfactory and reflexively append !g when the results are bad. Recently I've been doing that more on Google.

seanw444• 5 years ago

The bangs make it great. The keyboard interactivity makes it awesome. Being able to search, and then use the down arrows to select a result, and then press enter to follow the result, is super awesome, and I'm surprised Google hasn't gotten that right.

yosito• 5 years ago

I think it would be really great if DDG suggested bangs when you do a search. I often don't know which sites may be best for the search I'm doing.

potas• 5 years ago

For the last few years I've been trying to switch to DDG since I have already stopped using all Google services except search and maps. Until recently I was more often satisfied with Google results than DDG. Something has drastically changed in the Google's search algorithms in the last year or so and now I'm fully switched to DDG, as it gives me better results than Google, even for the basic search.

Sometimes I still try Google when I don't find useful results in the DDG but more often than not, there are no useful results in Google either.

Still, I don't know what should I do with Google maps, as I use it to find local businesses and when they are open. I don't use it for navigation at all, it's more like an address book for me. Haven't found a viable alternative yet. It's especially hard since I'm in a country that doesn't have good coverage in either Apple maps nor OSM.

skinkestek5• 5 years ago

I was the same until I started using Kagi. I was planning to force myself to use it for a while to test it out, but after two days or something I changed my default search provider and not because of politics but because of results.

It feels like going 15 years back in time - to the time when Google found what I searched for instead of mangling and/or ignoring my actual query every single time.

(Note: it seems some people consistently get better results. I'm not saying it is impossible to get good results with Google, only that I personally have given up.)

Edit: I might sound like a paid shill but I am not. I only know the Kagi founder via HN as freediver and I have not received anything, neither money nor anything else from anyone related to Kagi. (With one big exception: good search results :-)

jve• 5 years ago

Well, searching "Kagi" won't give you any results on Google, so would have been helpful if you provided url: https://kagi.com/

notreallyserio4• 5 years ago

IIRC I read somewhere that Kagi limits you to only NN searches per day, but I forget how low NN was. Low enough I didn't investigate further.

ncpa-cpl3• 5 years ago

What's a good alternative for recipes?

xqcgrek2• 5 years ago

Yes, and there's not a damn thing you can do about it. There isn't a credible alternative either, so the world is basically returning to the altavista epoch with online tabloids everywhere.

mey• 5 years ago

Give DuckDuckGo a try

bellyfullofbac• 5 years ago

I'm going to start calling it googlevista...

gumby• 5 years ago

Google remains pretty good if you’re searching for pretty obscure topics.

Which support’s Seibel’s point.

You do have to go to extra lengths to keep the search on topic though. Google tries to DWIM the search (“seems unlikely the user is really looking for insect embryology”). Is this well meaning or steering the search to revenue-generating topics?

Volker_W29• 5 years ago

I wish there was a hackable search engine where you could e.g. write a python script that moves all websites down that have ads or appear in some list.

geoduck142• 5 years ago

This is an interesting idea! Would you be interested in doing with with: Wikipedia, reddit, stack overflow?

rolph3• 5 years ago

anything that results from ignoring a specified word count from the query terms.

i.e. ignore 50% of the search query, go to the back of the line

Volker_W29• 5 years ago

Yes

hk__23• 5 years ago

You could achieve something similar with a browser extension like uBlacklist [1] with community-driven lists.

[1] https://chrome.google.com/webstore/detail/ublacklist/pncfbmi...

Volker_W29• 5 years ago

I know of uBlacklist, and it's nice, but I think python scripting can be more powerful than a simple Blacklist.

nikanj3• 5 years ago

I agree with the author re: results being useless, but strongly disagree on the motivation.

Google doesn’t show crappy results to optimize adwords, blackhat SEO hackers force their crap onto the fromt page.

The whole thread after the first tweet seems to assume Google is behind this, when in reality their failure comes from not successfully blocking the spammers

redisman3• 5 years ago

Oh poor Google. If only they had the resources to combat it

nikanj3• 5 years ago

” What would a paid version of Google Search results look like - where Google can just try to give me the best possible results and not be worried about generating revenue?”

The author seems to genuinely believe Google is not combatting it, but trying to generate revenue via showing lame results

mrtksn2• 5 years ago

I'm often critical of Google's recent product quality but how much of the blame is really on Google?

There are no longer "organic" content on the internet, everything is produced by professionals that are guided by analytics and optimisation.

The social media is searchable by it's vendor but the content there is also optimised for metrics that often don't align well with qualities like accuracy.

Only at places like HN or Reddit there's some organic content in form of commentary. HN is kind of special IMHO as its probably optimised for reach to a specific audience and pays for itself that way, therefore it can be optimised for quality through content moderation.

The web is well optimised for monetisation. Unless someone finds a way to optimize it for some other qualities, I don't think that Google or any competition can do anything about it.

donio• 5 years ago

There are no longer "organic" content on the internet

There is plenty of "organic" content, probably more than ever before. But there is also 100x more junk and Google is no longer good at surfacing the right stuff. Even for something as seemingly obvious as github issues it will often prefer spammified versions of the same.

mrtksn2• 5 years ago

Everything is measured and optimised by some simple metrics like profit, growth, reach or fame as far as I am aware.

Can you give some examples of great, contemporary organic content that Google fails to surface?

anonymous9023u• 5 years ago

spammified versions of the same

Seriously. What's up with these websites that are copies of Github/stackoverflow with a different UI? Cash grab for ad pennies?

Skylyz• 5 years ago

I'm pretty sure there is less organic content in proportion today than before, first because monetization wasn't in fashion, it was less common/accessible, but also because people who make organic content are fighting with SEO focused people who monopolize attention time / steal & rewrite organic content very efficiently. Less people viewing your organic content = less motivation to spend time on it.

If only we could send all these money obsessed intermediaries & creators to an island somewhere, left to scalp & scam & plagiarize each other out.

politician• 5 years ago

HN is a special kind of job board to steer people towards YC companies. You can tell this is the case because they regularly post job openings at YC companies that have comments disabled on the front page. Is this form of advertising bad? I don’t think so, but it’s good to recognize that there is a transaction here.

draugadrotten• 5 years ago

Google is not really fighting the political version of search results either. One visually obvious comparison is the searches for "happy black family" vs "happy white family". With such clearly politically adjusted search results, how much do you trust your search for "is the coronavirus vaccine safe?"

rightbyte• 5 years ago

What is the difference for you? I get "happy family" stock photos of either all white or all black skin color for both queries. The only difference is that there are like 10x more "happy black family" hits.

agumonkey• 5 years ago

It is an ironic if not sad fact of this global system. It's become its own metric with all the bias and larsen you can think of.

Nextgrid19• 5 years ago

There are no longer "organic" content on the internet, everything is produced by professionals that are guided by analytics and optimisation.

Then you detect commercial content and downrank it. Ads or the aforementioned analytics should be included in ranking signals to prioritize websites without them.

disgruntledphd23• 5 years ago

And then the partner manager raises hell about how you downranked their client, and it gets escalated, two VP's enter a room, one VP leaves and the penalty is quietly removed.

twoodfin• 5 years ago

Indeed. I suspect that, despite the sensibilities of the average HN’er, Google’s metrics tell them these garbage results are actually satisfying the average user more than results that alternative weightings would produce.

disgruntledphd23• 5 years ago

Yeah, but their metrics probably suck.

So do mine of course, and so do yours. Metric design is a hard, hard problem.

I've come to the conclusion that I should keep secret metrics from people so that they aren't captured by Goodhardt's law, but apart from the ethical implications, I haven't managed to implement this anywhere yet.

betwixthewires5• 5 years ago

There are no longer "organic" content on the internet...

There are billions of people on the internet. The likelihood that this is true is miniscule. Sure, it's probably smaller than it used to be, seeing as most people not in it for some commercial reason use silos to publish instead of their own websites, but if it looks like there's no organic results anymore, it's probably because your search engine isn't ranking organic results, because the websites are out there.

throwawayboise2• 5 years ago

And as long as Google is in the business of both search and advertising, that will not change.

They actually are not in the business of search, though -- at least not anymore. Search is just the hook to show the ads.

As long as the incentives are the way they are, things will not change and will get worse from a search quality perspective.

ClumsyPilot3• 5 years ago

I think this hits the head on tbe nail - its not a technical problem, its all about insentive

sixothree• 5 years ago

So the results listed higher all use google as advertiser?

alphabetting• 5 years ago

And as long as Google is in the business of both search and advertising, that will not change.

This has always been the case and general sentiment hasn't always been that search was bad. The only way they wouldn't be in search and advertising is if they charged for search, because ads pay for it. They know it is in their best interest to keep search results good so that people will keep using them and see ads. They seem to be losing the battle right now against SEO and marketing firms.

ghoomketu• 5 years ago

A thread like this happens every month now and the main point of discussion always comes to how companies like Pinterest, Quora, Stackoverflow clones, etc are openly gaming the search engine and making results shit (1)

I think first time I read this was like 5 years ago yet Google is doing absolutely nothing about it and these sites still dominate the results by gaming whatever metrics Google is using to rank them.

(1) https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...

ClumsyPilot3• 5 years ago

I was wondering for a while if SEO has gotten so good, or its Google that's gone to this - and I have a feeling that it's the latter.

Search for COVID rules for entry to moldova - for me and my friends the official website was like page 10 of search. Search for various government services, and most of the time top result is some scam.

This is inexcusable - is is to hard to prioritise official government websites? They could hire two dudes per country to index them all by hand and they'd be done in a couple weeks.

truculent• 5 years ago

Another problem with google search is that it's no longer something that you can become good at. Yes, the queries may have improved on some KPIs (although the link here would suggest otherwise), but you more or less get what you're given. It's very difficult to tweak your query to get better, or more specific, results.

rchaves• 5 years ago

Yes it had been getting worse, but I think there is a limit to “find relevant results” anyway, Google is good for objective answers, but for more complex stuff, there is an more and more answers, the web nowadays is overwhelming

At some point it starts reaching personal preferences as well, and just tracking me like Google does is not good enough for those filters, not to mention privacy concerns

For example, many many times it happens that I read an awesome article about a subject, then weeks later I try to google it back to show to a friend, and I simply can’t find it, because there is a gazillion other articles about the same topic

That’s why I’m building a custom search engine, where you index whatever you want, your bookmarks, your Twitter likes, building your own brain’s search engine

Stay tuned!

Volker_W29• 5 years ago

That’s why I’m building a custom search engine, where you index whatever you want, your bookmarks, your Twitter likes, building your own brain’s search engine

Link to the Repo please?

j45• 5 years ago

Google is likely inclined to return higher quality ad results, than high quality search results.

Clicking on one makes them money, the other does not.

Look at the percentage of the initial search result screen that is devoted to ads vs organic results.

142• 5 years ago

Any time I search how to repair X on my car I get 10 sites of places selling the part but none of the forums discussing how to do the repair. Back in the day it was not like this. Very sad and I have now learned to basically ignore the first set of results and scroll past them.

albertopv• 5 years ago

When looking for gift I go straight to Amazon. For travel booking.com, for info is wikipedia and so on. I basically avoid google as much as possible.

superasn• 5 years ago

This will always happen because there are people who make good content and then there are people who are good at marketing/spamming or have deep pockets to pay people who do.

Also people who are the experts in their field and produce great content are also terrible marketers and couldn't be bothered to create "backlinks" or whatever google wants from you to get your content on top. Which is why Google sucks to bad these days.

alphabet9000• 5 years ago

ive been noticing something specific that seems not good: sometimes if you search for a series of words as an exact match, sometimes it will match but other times it doesn't, depending on which string of words you use from an article.

e.g. lets say a webpage has the following text on it: "hello my name is john my favorite color is red" .. searching for "my name is john my favorite" might find the page, but searching "is john my favorite color is" will return 0 results; even though the quote exists and should turn up the same result

wayoutthere• 5 years ago

Yeah, Google today feels like the end stage of many of the pre-Google search engines. They were ok for a while, but eventually SEO tricks took over and ruined everything for everyone. What I worry about is that back in the 90s, everyone had a healthy dose of skepticism for anything you read off the internet. That’s not true today, and there’s a whole lot more incentive to put false information out there these days.

otherotherchris5• 5 years ago

Getting on the internet in the mid to late 90's required a fair bit of intelligence to set up a modem and drivers, "kids what the hell is UART", configure DUN settings, download and maintain browsers, etc.

I blame the iMac G3 and specifically Jeff Goldblum for all of this mess.

https://www.youtube.com/watch?v=A0QK0JfHzhg

PaulHoule5• 5 years ago

(1) No external competitor that is really better (e.g. DDG might make you feel better about yourself, but it isn't much better.)

(2) The internal competitor of advertising. If the SERPs were perfect you'd have no reason to ever look at the ad.

firebaze2• 5 years ago

This has been true for a long time. I think it is still safe to assume googlers are not stupid, so there must be a reason for that. My guesses, in order of decreasing confidence:

(by far) 1) globally seen, the results given are best for the general population

2) search/ad revenue from "still-google-users" vs. switchers is in a sweet spot

3) ???

I'd bet on 1), since google was historically nerd-focused, but that changed long ago, and including inertia and stuff, they have probably realized by now that the crowd of "i know what i want, just give me the link" is not driving key performance indicators.

jimrandomh• 5 years ago

Google is pretty bad at handling specific technical queries, and I'm pretty sure it's because their internal metrics don't account for the possibility that a query might have no useful results, or have only one result which requires some iteration on the search terms to find.

What happens is, if you search for something that's specific enough that there are few results or no results, it will either ignore keywords, assume that you meant to put a space inside a multi-word identifier, or spellcheck-correct something that wasn't actually misspelled. This produces convincing-looking decoy results, and you have to look closely (or click through) to find out that it's wasting your time, them rerun the same query in verbatim mode or with more quotes.

So then you've forced it to verbatim mode, and reached a query which, let's say, has one StackOverflow thread which you've already read and which failed to answer the question. Then your search results will be a couple pages of StackOverflow scraper sites. I never want to visit a StackOverflow scraper site. They should be easy to detect. Why aren't those domains being blocked properly?

dahart• 5 years ago

Also, how can a search category be SEO’d into ruin? Isn’t search engine optimization supposed to produce “better results”? Doesn’t Google exclusively control the results it displays…

This seems almost willfully naive. Why would SEO produce better results for the searcher? It’s optimizing rankings for sites who want to appear higher up in the results. Of course SEO is going to degrade the results. Of course every site on the planet is going to try to game Google search and appear higher in the list.

Yes there are conflict of interest issues between serving high quality search results and making money serving ads in those results, and yes Google is consciously allowing some of this to happen. BUT - this was inevitable. Any popular search engine is going to be gamed by the entire rest of the world, and the scale is too large for Google to control it. Michael might not be aware that, despite the conflicts of interest, Google really does spend considerable amounts of time fighting against search-degrading SEO?

bigodbiel• 5 years ago

Google broke it’s own algorithm pushing too far their agenda, and forgetting to balance the cat and mouse base with SEO.

The time is ripe for another search engine to dominate, preferably for niche segments of the web. I miss “I’m feeling lucky button” search button!

LaunchAway1• 5 years ago

Internet is just for tourists now. We all know how it sucks but no one finds a solution. Now what?

neals• 5 years ago

Google is slowly becoming BING. I would love for Mircosoft to step up and weird everybody out by building some kind of open source superior search engine.

hirundo• 5 years ago

Yesterday I heard a Joe Rogan guest mention a particular non-Rogan podcast episode. It was a discussion about science and policy by two professional, well credentialed scientists and a layperson. I Googled it, couldn't find it. Same for Bing and DDG. Turns out it had been a popular YouTube video, but they objected to the conclusions and de-platformed it, from YouTube and the big indexes.

It took maybe another dozen clicks to find it on one of the participant's own blog. And the podcast is still up and hosted by Apple. So it's something that you can find if you know about it, but not by searching on the topic. At this point, at least, the shadow ban is still soft.

Add that as a data point for Google no longer producing high quality search results.

mkr-hn2• 5 years ago

Apple hosts an index of RSS feeds. The hosting for the files is elsewhere and varies.

IronWolve2• 5 years ago

Yup, Video got removed from twitter and youtube. But could easily be found on Rumble and Bitchute.

When the video is originally hosted on Spotify on the JRE podcast.

Seems crazy why twitter and google feels the need to remove legal content

bmarquez2• 5 years ago

Rogan apparently* said that Spotify owns the rights to the JRE video, so they'll take down rehosted videos regardless of its content if they discover it. And if a video happens to be controversial by YouTube standards, it gets discovered faster and taken down faster.

*2nd hand source, but seems plausible

stargrazer• 5 years ago

Perhaps with all the deplatforming, delisting, and other deleterious effects, all the 'good' content is removed, while at the same time those removals are affecting their machine learning data sets. It is almost a self-imposed adversarial attack on result quality.

bsanr24• 5 years ago

I think that the model is, somehow, irreparably broken. Remember that, when it started returning photos of black people in searches for "gorilla", they just stopped using it for "gorilla" searches.

My suspicion is that its been poisoned by some combination of an improperly-considered tagging process and malfeasance.

ssl2322• 5 years ago

I find it pretty egregious that tech companies in particular consider a way to fight misinformation to be to make it difficult to find. Where's the trust in people to critically assess evidence? We seem too afraid they'll pick the wrong "truth". What does that say about our lack of respect for their ability to make decisions? Will us behaving this way possibly change their minds? Stupid, irrational people have always existed and will always exist. Better to at least try to train them to critically assess what they read - which will reduce total stupidity in the world - than to block them from reading such material which only makes them think there's a conspiracy. And, what better way to make high quality allies than to have them critically assess both sides and decide to pick yours? And, maybe we might even change our own mind having had a more complete body of information to draw upon - better for everything.

Blocking access to information seems optimally bad for truth.

glial• 5 years ago

If limiting the spread of 'misinformation' is their intended goal, it seems like they would have a bigger impact by just toning down their algorithmic recommendations. Leave the videos up and available to search, and stop doing hyper-optimized recommendations designed to increase 'engagement'.

Of course, that would lose them advertising money, which is probably why they didn't go that route.

willis9364• 5 years ago

I had this issue with a less controversial topic yesterday.

I was trying to explain VACnet someone and wanted to link the original talk. Youtube didn't have it anymore. 3kliksphilip covered the topic and he linked to the video. It said the video had been removed, though not why. 3kliksphilip's video helpfully said it was a GDC 2018 talk in the first sentence, so I searched google for that and found the video hosted on another site.

I don't think censorship was at play. I just think the world of mutable data is volatile and impossible to keep a 100% accurate up-to-date index.

grishka• 5 years ago

Searching anything related to programming inevitably yields some of those StackOverflow mirrors. Sometimes they are crappily machine translated into my native language. And sometimes they're ranked higher than the actual StackOverflow.

Google won't let people blacklist domains, so I had to write some uBO rules to get rid of those results on the browser side.

Volker_W29• 5 years ago

uBlacklist can blacklist domains on google

laurex• 5 years ago

I've felt this pain and often wondered why search engines don't allow one to set fairly complicated ongoing preferences that lead to more trusted results. Yes, I might miss some information by aggressive filtering but I'd prefer it to the utterly useless results I usually get.

I typically use DDG but it's gotten to the point where even putting quotes around terms still yields results without the term, and where there seems to be almost no way to avoid results that are untrustable SEO-driven dreck.

tonymet2• 5 years ago

Let's be honest, with rare exception, compelling & relevant content is no longer on the web. It's found on chats where people can be candid, like Signal & Telegram. To some degree it's on twitter where you can curate trusted publishers, and other social media. There are exceptions e.g. Substack, but most of the web content is seo clickbait.

The author ignores the more pernicious problem : ML fairness. Google is editorializing results by up-ranking positions it prefers. Check the results on any controversial topic like race, politics, abortion, covid treatments, etc.

The problem is that 95% of consumers believe google represents the truth.

Just go back to first principles before the internet and make sure you trust the publishers, sources & references. Also don't ignore your own observations. There's a PR campaign fighting your own critical thinking abilities.

Volker_W29• 5 years ago

The author ignores the more pernicious problem : ML fairness. Google is editorializing results by up-ranking positions it prefers. Check the results on any controversial topic like race, politics, abortion, covid treatments, etc.

Source/Proof?

temikus6• 5 years ago

Not the OP but I think he might be referencing this study from Columbia where they found that the “Featured” articles were mostly left-leaning: https://dl.acm.org/doi/10.1145/3290605.3300683

The cause of this is debatable though - be it editorialisation, unconscious bias on behalf of people working on the algorithms or some common qualities to the websites that lean a certain way politically.

Jweb_Guru• 5 years ago

Yeah, I often wish you could search public Discords without joining them... it's where a huge percentage of information lives nowadays.

omreaderhn• 5 years ago

It's found on chats where people can be candid, like Signal & Telegram

Can you recommend some of these channels?

Eiriksmal• 5 years ago

The point of those groups is that they are closed. Similar to how "Secret" Facebook groups became the late 00s equivalent of invite-only PHPBB forums (think what.cd). If you're part of whatever political or ideological movement is en vogue, you're free to candidly speak on whatever public platform you choose. If you aren't... You end up inviting a few like-minded friends to a Signal group to share your real thoughts and ask questions. They, in turn, invite a few more people. After a few months, you find yourself in a full-blown community; one where you can get all kinds of useful information.

But those communities aren't interested in advertising their existence.

psyc3• 5 years ago

The problem remains that Google no longer delivers the most compelling and relevant of what is still on the web.

lumost• 5 years ago

The comments seem to regularly reference the poor quality of content on crafting and other activities due to SEO spam and affiliate marketing. However I wonder if the root cause is a dearth of useful content which is also not trying to sell you something.

Anecdotally the only "interesting" content I find while searching is either from the old internet or medium. I suspect that many content makers have moved on due to lack of audience.

StreamBright• 5 years ago

Or high quality translations. It is actually really funny how broken translations Google translate produce.

honkycat• 5 years ago

A while ago I was doing research on guitar products.

I did what I usually do, open the first 6-7 websites in google, look at the quality of the website, and made a call which one looked the most reputable.

Only, here is the thing: All of the websites had different names. They had different domains. But, they were all very... similar. And then I started to compare them, and I came to a realization:

They are the same website, with the same content, with a different domain and slightly tweaked front-end. SEO has hit so rock-bottom, it is no longer good enough to be number one. You have to be number 1 - 10.

Similar story researching fishing equipment. Different domain. Same website. Same 10 products.

In both cases, I found a discord server and asked the enthusiasts on there for advice.

Traster3• 5 years ago

This is very common in high-frequency trading. There's only so much you can do to capture the number 1 spot, and the number 1 spot means x% of market share, so a perfectly legitimate strategy is duplicate your entire hardware 5x, since now you're not 1/10th of the leading edge, you're 5/15 instead. Especially when the costs of spinning up a new instance are low, but you're going a lot via a small increase in share.

vecter• 5 years ago

It’s more profitable to run identical clones of the same strategy N times?

alkonaut3• 5 years ago

Day one of "writing a reasonable search engine" would be to kill sitespam, no? Any time you see N sites with the same or very similar content, you should assume immediately drop the rank of all but a small slice at the top. You can choose to keep the one with most trusted incoming link (regular pagerank) but also you could just trust the oldest one in your index!

For example: there is now an epidemic of StackOverflow clone sites. They just post the SO answers with their own ads. But I don't want that site. So how on earth can Google show the clone sites on top of the true StackOverflow?

You'd think they have systems in place with hundreds of thousands of canary queries and "known expected rankings" such that IF one of the fraud sites manage to trick their system, they can just swiftly patch it to restore order and bury the clone sites after page 100. But no.

cheschire• 5 years ago

Yep. I lookup advice on things, find links to forums and discussion threads... the next search result is a different site with all the content from the previous site ripped off and filled into a new site template with different ads. And this is discussion threads!!

propogandist• 5 years ago

once upon a time there was a discussions filter/tab on Google and it was invaluable for researching products, places and things. Google got rid of the feature a long time ago and it's all optimized for SEO spam or Ads.

Discussions and forums have also been declining, as their discoverability has decreased dramatically due to SEO spam optimizations

Volker_W29• 5 years ago

Can you tell me the exact query?

mcv4• 5 years ago

I've seen that sort of thing a lot too over the past decade or so. These content farms aren't new, but still surprising. I can't imagine Google likes them, and they seem trivial to prevent: just compare the contents, and only show the best/most popular/most reputable site.

gsjbjt• 5 years ago

What are the usual ways to discover discord servers (also through search, or some other means)?

ehnto• 5 years ago

I notice this in car parts, 5 different websites all with the same catalogue. I think it's also how a lot of "influencers" start online stores. It's their face and brand but it's not their catalogue or inventory.

it is no longer good enough to be number one. You have to be number 1 - 10.

I don't blame them, a lot of online store's business model hinges almost entirely on SEO, and Google can change their ranking overnight for just about any kind of arbitrary decision on ranking signals. Building in some redundancy is a smart move, but it's a pretty big red flag that the ecosystem and incentives are all twisted up.

jmcgough• 5 years ago

I notice this a lot - when I'm search for information on a specific topic, or discussion around a specific error message from a library, there's several sites that scrape someone else's content for the purposes of displaying ads, and google doesn't always do a great job of delisting them.

The other thing I hate is "top 10" lists or "is x healthy" articles etc made by a writer who has to churn out x number of articles a day. They end up doing no more than surface-level research on said topic, the site or blog makes money from ads or referral links, and I waste a minute in figuring out that the blog is low quality.

Online ads have had such a negative impact on the quality of web content.

gombosg2• 5 years ago

For non-English content, it's similar, with auto-generated Google Translated trash content treated as more relevant than content originally written in the same language.

I used to struggle sometimes with DuckDuckGo but 90% of the time it now serves the same or better quality results.

Claude_Shannon• 5 years ago

On the other side, DuckDuckGo is bad at giving me results in Polish - so I use it only with English content. :|

markus_zhang• 5 years ago

Basically nowadays Google shows me Ads links on top, bunch of medium or content-grabber sites (think a website full of blogs from other blog sites).

scaredofgoogle• 5 years ago

They have us - that is, the wider internet - (by the **s) so why would they do anything other than return the most profitable results?

Wouldn't you do the same thing? (don't forget to toss billion-dollar scraps to competitors, to head off monopoly restrictions).

neuronic2• 5 years ago

I have recently researched a laundry machine. No chance - if you dont know where to ask or look you will be bombarded with fake marketing crap and affiliate cancer.

tptacek• 5 years ago

Google has never, ever produced high quality search results for medical questions. The jokes about this are about as stale as the ones about airline food.

redisman3• 5 years ago

Wouldn’t you need general AI for actually useful medical questions ?

mybrid• 5 years ago

Yahoo is the same as it always was, but these days it produces better results than Google.

Bing is okay but getting worse over time.

Google image search is the worst. Bing is marginal but at least useful.

ohmanjjj• 5 years ago

Google is no longer a search engine. It’s a propaganda tool funded by ads and run under the guise of providing results to search queries.

swalls• 5 years ago

I was thinking this last night, as I searched for where Android Studio installed NDK and was directed to an article that started from how to open Windows Explorer... with a whole bunch of how to troubleshoot opening Windows Explorer to pad the article for adverts.

didip• 5 years ago

I am starting to think that Panda in 2011 caused Google to take such a huge revenue hit that they never attempt similar move anymore.

littlecranky6710• 5 years ago

I wonder if the world needs a stackoverflow for products/travel/restaurant/recipies etc. Something with a reputation system that works pretty well in SO - i.e. if you are constantly ask stupid answers or given stupid/marketing answers, you get downvoted and at some point lose your ability to vote. Problem is of course, the majority decides what is "stupid", and to my experience going over Amazon Reviews, the majority seems not to get the idea.

Unfortunately SO doesn't allow questions a la "Which is the best printer for doing X" :(

littlecranky6710• 5 years ago

Given the helpful replies on things here in these threads by HN people, I think the problem could simply be fixed by using HN karma as a reputation system for such a site. We would have a community of supersmart and helpful people with access to lot of specialists in their field. And 3-6 months later, when marketing affiliates are catching up, HN would have its "eternal september" moment and HN would no longer be the place it is now, with upscale and kind questions and answers...

alecbz2• 5 years ago

I admit I'm somewhat pre-inclined to defend Google here a bit. That said:

I wonder how much of this is an issue of the incentives for producing different kinds of content, as opposed to just an issue of what Google chooses to optimize for.

I.e., yeah, lots of searches turn up mostly listicle bullshit. But is that because higher quality content is more difficult to monetize (e.g., people that are inclined to click on listicles are more likely to click on ads?), and therefore less likely to be produced in the first place, and even if produced, its authors are less likely to put in money/effort into SEOing it?

bromuro2• 5 years ago

At the initial stages of the internet we sere building quality content without even thinking to “monetize” - just sharing things we were passionate about.

Until people pretend to monetize it will never work like that again.

alecbz2• 5 years ago

What do you mean by "pretend to monetize"?

diegocg• 5 years ago

Google was still useful for things that are non-marketable. But in the last year or so, I have started to realize how awfully bad has gotten even at that.

Case in point: Open source mailing list archives. For various reasons, some times I have to search for entire email threads that are available in the web interfaces of some mailing list archives. I have part of a phrase of some email, I put that into google with quotes, and it returns some results, usually with the main archive being in the first results. In the last year or so, Google has started returning no results for some phrases of emails that do exist and are available in one or more public archives.

When that started to happen, I tried Bing. And Bing returns results with links to archive. So does duckduckgo most times. But for Google, it's like if that particular email I'm searching at that moment didn't exist (for other emails in the same mailing list it works fine). It only happens occasionally, but it's getting worse. So I have started to rely on DDG and Bing more and more, because they always find what I want.

So Google is starting to fail at some of the most basic aspects of a search engine, it's not just the ordering of the search results - there are some public web pages that it won't see for whatever reason.

betwixthewires5• 5 years ago

One thing I learned from the unreliability of Google search is that we should not rely on an external service to organize our information. No matter the service, it will change in some way, and you will lose access to some information.

ReactiveJelly• 5 years ago

Agreed. "Indexing the world's information" sounds like something that can't go wrong.

It's only in the last year of trying to index and organize information within my own company that whoever controls the index holds a _lot_ of power to index it poorly.

SpelingBeeChamp2• 5 years ago

What do you suggest as an alternative? Index the internet yourself? Web search as a public utility?

visarga6• 5 years ago

Why are browsers so forgetful? Local text search is an easy problem at single-user scale.

I am working on a browser extension to automatically grab text (visible phrases) from the pages I am reading in the browser, save them to my index and implement search. I want to always be able to recall anything I see. In order to match texts I will use neural embeddings computed with a pretrained Huggingface model, fast enough even on CPU.

An alternative way to implement this would be to take screenshots every few seconds (only after recent user interaction). Then I can OCR the images and search anything that passed on the screen, in any app.

It's as if I become the Google Bot responsible with exploration and filtering of potentially useful data.

In the future I might be able to use my index for neural question answering, add a voice interface to my data.

Of course the local index could work for images too. You can grab images and embed them with CLIP to get Google Photos like semantic search.

I think web browsers are going to morph into personal internet agents, responsible with logging, searching, interfacing with external APIs and services, filtering ads, news and mail, making sure we don't accidentally leak personal information, etc. In other words AI that works for the user, locally.

noizejoy4• 5 years ago

I for one am grateful that some of my worst mailing list accidents (like accidentally sending a privately intended message to a mailing list, rather then an individual email address) are starting to disappear from Google’s relentless claws.

hubraumhugo5• 5 years ago

Curious to hear: what are some rising Google competitors that try to do a better job?

XoS-490• 5 years ago

https://you.com

JSONderulo• 5 years ago

I've been playing around with you.com after someone mentioned on HN. I dig the layout/quality of results. It's kind of interesting to have it feel like you're netflixing your search results by source (reddit, stack overflow, etc.) I'm still toggling back and forth on you to google because i feel like i might miss out something. Surprisingly some the results have exceeded my expectations. We'll see.

jart• 5 years ago

It's hard to compete with Google. Back when Google started, you could fit the world wide web on a single computer. Today you can only really get started the same way Google did if you focus on an unfancy subset of the web that hasn't been react'd and wasm'd into exabytes of opacity. https://wiby.org/ does it I think by refusing to index any HTML page that has <script> tags. Websites like https://millionshort.com/ provide pretty nice results sometimes, but I don't think they actually index the web on their own; they probably use the Bing or Google API on the backend. With million short if I want bike reviews for example, I get the authentic blog posts from 2010 with amazon affiliate links to bike listings that are no longer being sold lol.

The problem is I don't think the Wiby model (rejecting JavaScript) or the MillionShort model (rejecting popularity) have done a good job capturing what we loved most about the old web and systematizing its curation. There's definitely an opportunity for someone to come along and create a focused niche alternative that's better.

One way I suppose it could be done, is if you convinced a bunch of trustworthy high status trendsetter type people to subscribe to a paid service with a browser toolbar that lets them click a button for each website they visit to say "I like this" or "I dislike this" and then use that information to train a neural network that divides seo spam from content. Mix that with classic page rank and you might have something good. I'm not sure if it'd ever appeal to a more general public audience though.

Anyone who does it is also going to want to make a deal with the archive.org to somehow get a snapshot of the old web, to recreate those original experiments. Or possibly even resurrect an old build of it. Plus Gutenberg. There's enough content from 2005 web alone and all the books published before to last anyone several lifetimes. That's actually one of Google's blind spots. They're so good at up-to-the-minute indexing of current events that sometimes if you just want to get the text to something like Seneca it's like pulling teeth.

ColinHayhurst4• 5 years ago

https://www.mojeek.com/ no-tracking, independent crawler/index

1_player2• 5 years ago

Brave Search is actually quite good, and better than DDG in my opinion. And it's their own index, not piggybacking on Bing or Google.

Axien• 5 years ago

Siri, Alexa, and Amazon.

When someone need to know the weather or a sports score they usually just ask Siri or Alexa. If they want to do product research most people go to Amazon. Google search is now below 50% and still falling.

Think about it. You want to buy a coffee maker. What is the first site you go? It is probably not Google.

donkarma• 5 years ago

kagi.com

srcreigh2• 5 years ago

Blacklisting non advertisable media from the index. That's news to me. Hmm.

Why? Cost savings? Missing hard dependency on marketing ops knowhow? Utter domination of non-marketing material? Forcing experienced privacy-conscious users to a different platform to reduce bad media coverage?

dannyw• 5 years ago

Monkey run experiment, revenue goes up, monkey get promotion.

nullc2• 5 years ago

Likewise, I constantly find that google won't turn up mailing lists posts and forum posts even when I have a fragment to search for exactly.

Some investigation seemed to suggest they were hard excluding any page which hadn't changed in ~5 years.

I have to wonder if the poor behavior from google doesn't come from automatically a/b testing behavior changes against advertising revenue. Up to the point that you make users give up completely giving worse results will result in more search traffic...

r721• 5 years ago

Some investigation seemed to suggest they were hard excluding any page which hadn't changed in ~5 years.

I had another hypothesis: that they exclude pages which have no backlinks from other websites, and at the same time are hard to reach from front page of the website they are on. It seems old unpopular mailing lists posts and forum posts are exactly like that.

Commodore63• 5 years ago

Google seems to value recency over quality - a huge shame, given that the quality content has moved off the open web into walled gardens. If you elevate recently, you get churnalism, blogspam, and vacuous GPT3 bot content.

wolpoli• 5 years ago

Has anyone noticed that we can't click past page 10 on many search queries? Isn't Google being misleading when it claims it found 2 million results, but it won't show past result 100?

everydaybro• 5 years ago

I have a question: why is google still number one? everyone know that the search results are bad

throwawayboise2• 5 years ago

Inertia. It's the default on most browsers. Most people don't really think about it. "To Google" has become a verb, that's how entrenched it is.

jeffybefffy519• 5 years ago

Based on number of search engines appearing on HN recently I suspect disruption is around the corner.

flenserboy• 5 years ago

This has been true for over a decade. What's changed is how obviously awful the results are.

mark_l_watson• 5 years ago

I agree that Google search is much less relevant to me than it was > 10 years ago. I love some Google products (GCP, YouTube Music, Play books/movies, and paid for no advertisements YouTube) but search is no longer one of them.

That said, Google search works better for me if I use a private browser tab so the results don’t depend on search history. I find DDG to be useful. One good use case for Google search in the logged in mode is when I am searching for work related things that I might want to influence what I see in YouTube, but I could simply search in YouTube.

wmu• 5 years ago

DuckDuckGo is also not good. Last week I switched to Bing and its results are surprisingly good.

ra-mos• 5 years ago

Could the issues be less presumptuous about googles business, and more technical? E.g the incorporation of NLP model(s) in search that don’t perform well in the wild?

Curious about their query parsing as well. I can’t recall exact queries, but I had instances where scrambled typos (but still obvious words to me) broke the results all together & no suggested fixes.

throwaway4good• 5 years ago

Search doesn’t really that much anymore - not even to Google - their monopoly on advertisement brokerage does though …

ummonk2• 5 years ago

I suspect these ad-ridden junk sites are optimizing more for Google than for Bing, as I tend to run into them more if I try to check Google's results instead of relying on Duck Duck Go.

dijonman2• 5 years ago

Google and their shareholders are addicted to ad money. Personally speaking I think this is more harmful to society than cigarettes.

I hope we normalize this and start fighting back. Ads first destroyed youtube, google, and I’m sure a bunch of other products I am not thinking about.

Enough is enough, and we need to decouple morality from content and edge back to the anti censorship wild west mentality the internet championed for so long.

zmmmmm• 5 years ago

Is there some special significance to this individual? There doesn't seem to anything more to this than their personal anecdotal experience and opinion.

seaman1921• 5 years ago

Nope, just an excuse for HN big-tech shitposters to crawl out and post their own anecdotes and theories without any link to a proper research/study with concrete data.

99_002• 5 years ago

On the one hand, I'm sad that other people are realizing this because it means they can fix make some better before I can.

On the other hand I probably don't have the ability to make something better so it's good that someone will come up with a solution I can use.

uejfiweun• 5 years ago

Certainly Google Images has taken a major dip in quality. I am unable to find images that I could even find a few months ago, and the number of results is shrinking for identical queries. On top of that, they are rolling out an absolutely HORRIBLE interface for image search on mobile.

Google is just in "big company" mode where the company is just so vast that it is nearly impossible to guarantee a consistent product quality. It will continue to offer good salaries and nice perks, but the innovation has stopped, and the company is doomed to inevitably slowly stagnate and fail.

tester756• 5 years ago

how do you people remember the quality of Google years ago?

If somebody asked me whether Google's quality was/better 10 5 3 2 years ago then I'd have no idea despite using it daily shitton of times

rickdeveloper• 5 years ago

I actually created "my own search engine" because of this: notrashsearch.github.io.

It uses Google search tech under the hood (which I've found superior to other search engines), but filters results with a white list. It's only ~100 sites long & very focussed on STEM, but the results are surprisingly good.

If anyone has suggestions for site to add, please let me know!

tome2• 5 years ago

All search engines lack a feature that would make them hugely more useful: disambiguation. Suppose I search a common name "John Doe". I should next be presented with a disambiguation page allowing me to select whether I meant "John Doe who was President of Calexico 1905-1909", "John Doe who won the World Series with the Greensocks 1975" or "John Doe professor of Spanish Technicalities at Idaho Institute of Science". It shouldn't be my job to disambiguate my query. The search engine knows much more about the total search space than I do!

colordrops• 5 years ago

I vaguely recall google doing this long in the past. Probably just confusing it with Wikipedia though.

99_002• 5 years ago

Not just search. I subscribed to a print news paper.

I enjoy reading the news again. Good mix of local, financial, entertainment, international, human interest.

I'm exposed to things outside my bubble and I often like them.

The print edition excludes a lot ofnlow quality content because its limited space.

dageshi• 5 years ago

I've noticed the same thing and given it some thought recently.

I think a structural shift has occured. A decade+ ago people made sites about their hobby's and interests, they were amateur experts on subjects and because they enjoyed what they wrote about they tended to build up extensive knowledge bases on their subjects, effectively their sites let you learn a subject rather than just trying to give you quick direct answers which might actually not be right for you, their sites let you understand what you really needed.

I think the thing that changed, is those people or their newer versions moved to youtube instead. People make videos instead of articles and what's left on google is the seo'ified crap that lacks the deep knowledge and context the original sites had.

Of course google owns youtube so it likely doesn't care, but yes google search is now a lesser product and I'm not actually sure google can do anything about it.

ALittleLight• 5 years ago

One point about all the "Top X lists" and general blogspam that Google returns - I had once created a website that I was trying to get listed on these sites. I emailed a few of them to get my site listed and they transparently and directly offered to list my site for relatively small amounts of money. It was literally an exchange of 100 dollars for listing my site at number 1 and I got to write the description that went on their website.

My point here is to add that in addition to the Google results being bad (they are listicles and blogspam rather than the answers you want) they are also corrupt in that people can just pay the owners of the spam sites to get the listings they want. This is a tax on the users by giving them indirect and bad results and also a tax on creators by forcing them to pay third parties to get their websites to rank in the appropriate Google searches or work on SEO voodoo to rank themselves.

pastelsky2• 5 years ago

I wish Google had a toggle switch that isn't slowed you real user generated content... Like filter out all sites that are in Alexa top 10 for the category, and just show geniune niche blogs and forums with rich informed discussions.

stavros• 5 years ago

Google sucks completely for me (I use DDG as my main search engine but that only sucks slightly less). It's all spam and I can't find anything. In particular, I used to be able to find useful blog articles on any programming problem I had. Now it's either SO or nothing, whereas I'm sure the articles exist somewhere.

I saw kagi.com mentioned on an article here and tried it out, and so far it's been much better than Google. It gives me reddit results, which are very useful because it's just people posting their reviews/solutions, and it gives me small site results, which are usually helpful. If I have to pay for it, I will, because the free option is just useless now.

rc_mob3• 5 years ago

marginalia is the way other one Ive been using lately

spankalee2• 5 years ago

I'm a current Googler, so yeah, I'm biased, but I work far away from search so I'm basically a plain consumer of it, and this is lazy thinking:

"I’m pretty sure the engineers responsible for Google Search aren’t happy about the quality of results either. I’m wondering if this isn’t really a tech problem but the influence of some suit responsible for quarterly ad revenue increases."

I'm pretty sure Google doesn't make enough money on third party sites to intentionally make its own search results worse.

What's happening here is the ever-growing battle between search algorithms and SEO. Most of sites that he is complaining about are likely doing an incredible amount of optimizations for search engines and human psychology to show on search results pages and get people to click on them. They A/B test, within singe sites as well as run the same or slightly modified content through site networks.

So sites optimize to get crappy filler content on Google, and Google changes to demote those sites and produce better results (which people still complain about).

This is also the reason that it's not so simple to do better than Google. A new search engine also has to have an algorithm and presumably it'll share many of the same approaches that Google has used and have been gamed by content farms. If a search engine does come up with a break-though mechanism to separate the bad from the good, then either sites will adapt to that, and/or Google and other engines will adopt similar mechanisms too.

And if a search engine somehow made an un-gamable algorithm, then that would be a pure good for humanity and go them.

But also in these types of discussions you really need to bring receipts. Otherwise it's hard to talk about what's even good or bad. What terms did he search for? Which results were bad? What should have been there instead?

I did a quick search for "hip replacement" and the results look great to me: top result from American Academy of Orthopaedic Surgeons, a definition card, then Mayo Clinic, medlineplus.gov, Johns Hopkins, local MDs, new stories, images, WebMD, etc., and seemingly useful related searches like "What are the signs I need one"...

Maybe that's just not a monetizable enough term. "quit smoking" should maybe turn up crappy help articles, but it's also pretty good. Two ads at the top this time, then CDC, Substance Abuse and Mental Health Services, lung.org, local results, WebMD, etc..

Not that I don't believe the author, but he's certainly invested either directly or indirectly into companies working both sides of the SEO war, from search tech like Metaphor to algorithmic SEO like RankScience. So rather than trusting him that results are bad, it would help an honest discussion to point out examples.

otherotherchris5• 5 years ago

Google strikes out the least common (and therefore most important) keywords in my queries.

Google replaces specific jargon with common words that are similar in meaning to laymen.

Google returns pages which do not contain any of the keywords that I've searched for at all.

If you use your example and search for "hip replacement", or any "what are the signs" variation, you get all of the content free SEO spam farms specifically catered to returning useless garbage to medical queries, but no informational academic/government/ngo health sites.

If Google's AI is catering to the GPT-3 and future GPT-X SEO adsense spam vendor market and Google is not curating a collection of high quality reference sites like librarians have been screaming at them to do for 20 years, then it is doomed. It is nearly completely broken now, and will become more broken every year going forward as the algorithm/counter-algorithm fight continues and human content drops off the index altogether.

spankalee2• 5 years ago

If you use your example and search for "hip replacement", or any "what are the signs" variation, you get all of the content free SEO spam farms specifically catered to returning useless garbage to medical queries, but no informational academic/government/ngo health sites.

I did do the hip replacement search and got no spam and all medical organizations.

I just tried "what are the signs I need a..." and chose the first completion of "pacemaker". That also had very high quality results from government health orgs, medical associations, Mayo and Cleveland clinics, healthgrades (looks ok), and a nice list pulled from Vein, Heart and Vascular Institute.

Do you have a specific search that has bad results?

hereforphone• 5 years ago

Ten years ago I complained on a random forum about Google's seeming refusal to give me information about running Skyrim under Linux with the Wine emulator. I tried various search combinations and all I could easily get was information about wine manufacturers and locations in the Skyrim world. Didn't matter if I used "emulation" or "wine emulator" or similar terms, even in quotes, or if I used the '+' character, or otherwise tried to harness Google's literal search functions.

Was I searching wrong? Didn't seem so to me at the time, but I don't remember the exact queries. Those in the forum thread thought I just didn't know how to use a search engine. I still think it was Google telling me what it thought I really wanted, instead of what I was literally asking for.

I still see results that don't include words that I search for in quotes, often.

visarga6• 5 years ago

I found that it confuses subject with object: do you want A for B or B for A. It's all the same. Maybe Google doesn't really use neural nets in ranking, or they are a joke.

Desktop web search is trending down, voice and mobile interfaces up. They should be focusing hard on direct question answering based on retrieved web pages. A recent paper by Deep Mind shows how it can be done, not that it matters, they won't jump on it. Web search should be just as smart as Codex, adapting and combining knowledge for the user.

https://deepmind.com/research/publications/2021/improving-la...

hwers2• 5 years ago

Just today I googled "torch normalize -1 1" and it said "zero documents matches your query". Insane stuff. (Censorship with crazy high false positives? Incompetence?) I went to duckduckgo and immediately found a useful answer.

anarazel• 5 years ago

Isn't that just because -1 is treated as a negative match? Which you then also request as a match?

modeless• 5 years ago

Um, just in case you don't realize, "-1" selects pages that don't include "1". You just asked Google for pages that both include and don't include "1", and that's the empty set. I'm not aware of any way to search for negative numbers.

This is really a UI problem. Google should probably use the context in this query to infer that you didn't literally want to exclude "1" from your results. However, half the comments here are complaining about Google not taking their queries extremely literally. This problem is a good demonstration of why inferring user intent can be useful and taking user input literally all the time is not actually a great idea.

melenaboija• 5 years ago

I tried just to see what is returning for just '-1 1' and is able to understand it, as latitude and longitude. But I use 'minus' for these cases.

omegalulw• 5 years ago

Try torch normalize "-1" 1

It gets me to https://pytorch.org/docs/1.9.1/generated/torch.nn.functional...

Note that that will only normalize to (-inf, 1]. Tweaking the formula on that page You want (v - vmin) / (vmax - vmin + eps) with vmin and vmax produced along th dimensions you care normalizing against.

temikus6• 5 years ago

Yeah I often have to add “reddit” afterwards and hunt for links there or use “site:” predicate to get decent results in addition to using a results blocker to filter out a lot of useless SEO results (e.g. Pinterest/Gitmemory)

hartator• 5 years ago

I wonder if it will make sense to now build a meta search engine using SerpApi [1], weight couple of websites more like reddit or stackoverflow, remove adds, and repackage all this listings into a super simple UI.

[1] https://serpapi.com

umvi2• 5 years ago

Ugh, this is so true. It's getting increasingly difficult to break out of the SEO twilight zone. SEO is the worst thing that's ever happened to online search because it's completely poisoned the top N search results with garbage/ads. I have a very hard time finding organic content anymore. For example, I was trying to find an organic (written by a human who has tried the two products) comparison between two processors. It was impossible - I could only find AI generated sites that programmatically pulled the specs in a side-by-side comparison (which I had already done) but offered zero additional insights.

short122• 5 years ago

They wll show you 49 YouTube results though facepalm

Shorel• 5 years ago

This is true.

But, the important issue is that no one else is producing high quality search results either.

When this changes, Google can kiss his profitable search engine goodbye. But this is not an easy engineering challenge.

hashtones• 5 years ago

Too bad page rank became the bastard child of SEO bots instead of the novel, mathematically innovative tool it was.

NaturalPhallacy• 5 years ago

I ditched google (including my alphabet stock) a few years ago.

They're become so big and bloated they think they know better which is death to innovation.

DDG is my default search engine one every device and has been for ages.

kelnos4• 5 years ago

Also, how can a search category be SEO’d into ruin? Isn’t search engine optimization supposed to produce “better results”?

No, SEO is supposed to get your particular website higher up in the rankings, regardless of whether or not the results are better for users.

"SEO" is just a nice marketing term for "figure out how the search engine works and trick it into listing your site higher". It's learning what metrics a search engine prioritizes, and then playing to those metrics.

thehappypm• 5 years ago

Easy to blame some “suit” but when the rubber meets the road, Googlers want their RSUs to be valuable maybe more than any other group. Zero incentive to put user experience first at any level of the organization.

whatgoodisaroad• 5 years ago

To what extent is this a supply side problem? What's the counterexample high quality health website that Google should have included?

Part of the problem is surely how Google's ad model influences the success or failure of various kinds of websites, but a deeper problem seems to me how anybody qualified to share medical information freely would be acting against their own interest.

mkl95• 5 years ago

Google have found a sweet spot where their search results are almost correct but still insufficient.

That "near-correctness" ensures you don't lose hope of finding what you are looking for, so you keep reading result after result. And most of those sites are conveniently loaded with ads, sold by guess which company - Google!

Eventually, pagerank cashes out, and you find what you are looking for. You receive a much needed endorphin kick, and you go on with your day. Rinse and repeat.

pxmpxm6• 5 years ago

I would bet a large amount of money this is the correct answer.

It certainly is the approach facebook has taken with their marketplace place search engine, where the signal to noise is just barely high enough to keep the audience and there is absolutely no tooling exposed (verbose search etc) to increase the signal rate.

u2077• 5 years ago

I’ve noticed they started to limit search length, my search was too long and they removed small words like “of” and “the. I also saw a message somewhere along the lines of “to show you the best quality results, we limited this search to two pages”

Google is only good for finding stuff you already know exists. What ever happened to exploring the web?

csours• 5 years ago

Web search is an adversarial context. Just try to list the things a search engine provider has to fight against, then try to define some criteria for each of those things that more than 50% of the population will agree upon.

a_square_peg• 5 years ago

I wonder how much of this is due to Google or that the rest of the web (the crappy parts at least) has figured out the algorithm sufficiently to degrade its performance overall?

paulcole• 5 years ago

It feels like I’m the only one who gets consistently excellent results from Google search (and it feels like they’ve gotten better for me over time)? Maybe it’s because most of my searches are basic and lowest common denominator? I just don’t need obscure information most of the time and when I do it’s honestly still not that hard to find it.

tucosan2• 5 years ago

Since the incentives are currently stacked against the user, we might need something similar to AdBlock for search results with a community managed blacklist.

I personally use uBlacklist [1] to black useless sites like Pinterest or wikihow to appear in my search results.

[1] https://chrome.google.com/webstore/detail/ublacklist/pncfbmi...

aronpye2• 5 years ago

Is there a plugin which allows you to curate your own index, i.e. a whitelist of sites to search?

DoingIsLearning4• 5 years ago

With all due respect to Michael but has he not used Google in the past years? This has been an issue since at least 2016. SEO won.

Maybe some googlers can answer this but i assume that Singhal was very conservative against AI for ranking. But when he left Giannandrea started rolling out the "natural language" queries with "let me just ignore your query because we know better" algorithms, also slowly removed operators/keywords used by power users. And well SEO really have gotten better so it all piles up.

bravoetch• 5 years ago

This has been a known issue for years already. It's the 'attention economy' where nothing matters as much as engagement. Quality is not even an important metric. What we've been left with is visiting the same few websites we like, interacting on a couple of non-corrupted social channels where ads can't invade, like group chats in signal, and constantly unsubscribing from email lists because even the local ice-cream store knows that mailing lists are the best conversion channel.

baxuz2• 5 years ago

I've tried searching for some technical details on some of my car's components. No matter what search query I used, the first few PAGES of search results were links to non-original parts on ebay, amazon, aliexpress etc.

To get the actual details I had to go register on a forum and ask people there. Feels like pre-google all over again.

woodruffw2• 5 years ago

Has anybody else noticed a decline in Wikipedia's placement on many Google search results?

More often than not I want Wikipedia as my first result, both as a cross-reference for anything else I click on and as an index for other useful or interesting links. They were consistently in the top 5 results for anything that actually had a Wiki page for years, but now I have to pull the top few pages or even write `$query wiki` to get Google to reliably bring Wikipedia up.

temikus6• 5 years ago

Yep. Have to do that with medical/chemistry queries all the time.

Noumenon723• 5 years ago

If you know you are trying to search Wikipedia, go to chrome://settings/searchEngines and add a search engine with keyword 'w' and query URL 'http://www.wikipedia.org/search-redirect.php?search=%s&langu...'. Then you can type 'w whatever' to search Wikipedia for 'whatever'. Firefox has similar feature called "Add a keyword for this search".

woodruffw2• 5 years ago

I'm not trying to search Wikipedia -- I'm trying to search on Google and use Wikipedia as a cross-reference.

sparkling3• 5 years ago

Yes, can confirm.

KingMachiavelli• 5 years ago

It's a bit ironic that this post is part of the problem. It's very hard to make money in decent product reviews but there is money in SEO optimization & referral links. Most product feedback and honest opinions are shared on sites like Twitter & Reddit which range from bad-SEO (Reddit, Youtube) to deliberately anti-SEO (Twitter, Instagram, Pintrest). While this thread/opinion is on HN ATM, it will quickly be gone and no one using a search engine will find it (if they do it will probably be via finding a HN or Reddit post first).

That said I'm not sure what search results this person is actually getting but I really haven't had a problem using DDG or Google. I just know that what's marketed and used by the average consumer tends to be pretty bad if you ask any expert or enthusiast. I don't think this is search engines being bad at their job it's just that most topics bifurcate quickly into average people and amateur experts/enthusiasts.

Here's an example. Coffee. It's a very common beverage that millions if not billions of people drink every day. However, coffee culture for the average person is very different from enthusiast coffee culture.

Search engine's are not even that bad... if you look up a good coffee grinder. The top sites on both DDG/Bing and Google do mention the difference between a blade and burr grinder. The most recommended option looks pretty decent. This is probably a much better option for 90% of people than getting the 'enthusiast' (no-brand industrial burr grinder) option off of Ebay.

Anyone who complains about search engine result quality is a completely different demographic from the typical person. I don't think Google's search as gotten worse but rather typical user (i.e. training data) becoming more average.

Another good example are TVs. Everyone and everyone's dad knows that Costco has some pretty good deals on TVs as well as offering a best in class warranty. However, if you read the forums and in-depth reviews you will quickly notice that Costco doesn't have 90% of the best TV in each price range. The average consumer wants the best Costco TV not the best TV as long as you get X% discount and you calibrate yourself, etc.

betwixthewires5• 5 years ago

Anyone who complains about search engine result quality is a completely different demographic from the typical person. I don't think Google's search as gotten worse but rather typical user (i.e. training data) becoming more average.

I hear this argument a lot, and I think it is a cop out.

Google literally bills itself as being able to personalize itself to your interests and being able to learn based on your queries. On top of that, google used to work 7 years ago essentially perfectly. If there were really such a divergence between myself and "typical users" you'd have heard them all complain about google then, because they were using it also.

What's going on here is a misalignment of incentives between users and Google. They don't have an interest in being useful, they have an interest in herding eyes to the most profitable places and in using people to train their AI. They believe nobody can oust them as incumbent and so they no longer feel they have to deliver a good product if doing that hurts the bottom line.

bryguy32403• 5 years ago

300+ comments in a hour??? C'mon now, this all can't be genuine. I don't think Google returns the best results all the time either, but most "outage" posts don't get this much engagement so quickly.

Zamicol2• 5 years ago

All last week I was lamenting to my coworker at the decline of Google for technical results.

Anecdotally, I googled "the oa wikipedia" last night and Google had no clue what I was talking about. This morning it's working as expected.

It was ironic to be confirmed with pop culture and a Hacker News post.

I've noticed a decline in the last 3 years, and a huge drop off in quality in the past few months.

Another example from a few months ago was searching for "milankovitch cycles" and Google had no clue. A few days later it was working again.

kvhdude• 5 years ago

(circa 2012) i bootstrapped a company that connects people searching for information to providers of said information in a real time chatroom created for the duration of the query. This is to solve pogo sticking when the website information is too dense (say search for quitclaim deed without knowing too much about it). I failed to get enough users on both sides ('two sided market'). I am not from search/web space - my expertise is in building routers/switches in the 90s. I extended xmpp so that you could query from any chat box that can interwork with xmpp.

mrdrozdov2• 5 years ago

I don't believe this (yet) and here's why.

The claim is that Google search is producing worse results than in the past. The analysis is mostly anecdotal, and similar claims have been made before in a more concrete way. A prime example is "time to cook onions" giving incorrect results, covered in this slate article: https://slate.com/human-interest/2012/05/how-to-cook-onions-...

What we need is to see is specific queries, the results returned, why they're wrong, and what they should be instead.

mrdrozdov2• 5 years ago

Some folks should post comparisons between Google and other sites such as:

* https://bing.com

* https://duckduckgo.com

* https://you.com

Maybe use the categories mentioned in the twitter thread such as health, travel, recipes, product reviews, etc.

thebetrayer• 5 years ago

I noticed this too over the past couple of years. Google, Alphabet, and its shareholders don't care because they are still making so much money. I truly think it's by design to further ruin the state of the internet, just my opinion.

Has anyone else noticed how many important websites, such as news organizations, have Taboola or similar ads? Like it's the only way to make any money online. Sensationalized paginated joke content with ads on every page.

It's crazy how many websites are scrapping stackoverflow and getting on the first page of google results. Like, is it hard to check if identical content is on stackoverflow? lol it's not hard.

hn_throwaway_99• 5 years ago

I agree with this 100%, and could totally admit that I have a giant deficit of imagination, but I still have a huge problem imagining how a company could disrupt Google search at this point (unless that other company is Amazon, Microsoft or Apple, i.e. another company with gigantic resources).

The costs of setting up a comparable search engine these days must run in the multi-billions. The Internet is huge now, so your crawling and indexing costs would be giant. Not to mention that it can actually be hard to crawl many sites that attempt to restrict indexing to "the big boys" of Google, Bing and a couple others. Also, Google controls the primary on-ramp of Android and Chrome. They also control the primary ad networks that make running a search engine profitable.

I compare it to Microsoft. Nobody really ever "disrupted" Microsoft in the desktop OS space - they still have a greater than 75% market share. The thing that changed, of course, is that the "desktop operating space" as a category became much less important with the rise of mobile, and Microsoft famously lost that OS battle. So my point is that I find it hard to see Google being "disrupted" when it comes to the "Internet search market" - the only thing I really see being possible is if some other technology came along and superseded "Internet search" as a category.

visarga6• 5 years ago

the only thing I really see being possible is if some other technology came along and superseded "Internet search" as a category.

Free natural language search, based on dialogue. A cross between GPT-3 and web search. (https://deepmind.com/research/publications/2021/improving-la...)

turrini• 5 years ago

For better search results, I always use google in verbatim mode coupled with this TamperMonkey script to block scrappers and other stuff:

https://greasyfork.org/en/scripts/1682-google-hit-hider-by-d...

throwawaysea• 5 years ago

It’s not just about high quality search results. It is also about wanton manipulation of societal narratives, which ultimately is manipulation of elections. For example, YouTube recently started taking down videos of Joe Rogan’s interview of Dr. Robert Malone. One of the concepts Malone brought up is ‘mass formation psychosis’, which has resulted in many people searching for that phrase. Google has a dystopian message at the top of their search results that suggests they are manipulating the search results or in the least using such messages to undermine the legitimacy of certain content (https://twitter.com/JackPosobiec/status/1477403661701689352). This is not how a search indexer should be operating and it is not the basis for open societies where you can freely exchange ideas on an even playing field.

lehi• 5 years ago

You shouldn't expect DuckDuckGo to be better. DDG results have also gotten even more terrible with time.

I realized just today that adding quotes around search phrases in DDG makes it return nonsense (3 screenshots): https://imgur.com/a/2SHpjPG

nyx_land• 5 years ago

I often have to search the web for information about hormone therapy for trans peoole, long story short being that doctors tend to be pretty misinformed about HRT for trans people because the standards they use are based off information that is 20 years old, so trans women in the know tend to do their own research in order to get a decent HRT regimen.

Google is bad in general for health stuff, but it's particularly bad for searching anything trans-related because almost everything that comes up will either be clickbait liberal feminist listicles or clickbait right-wing transphobic FUD. Either way it's completely irrelevant to searching for something like differences in administering estradiol valerate vs estradiol enanthate. Like other people I tend to just look on reddit to find stuff that isn't SEO'd to hell, and there's a big community of trans biohackers on reddit, but it's very worrisome that it's this hard to find good content on the web without looking on yet another platform owned by a corporation that siloes peoples' content and can delete or mismanage that data at a moment's notice. It would be a tragic loss if reddit suddenly decided that the TransDIY subreddit violated the TOS for some reason, and I could very well see that happening.

It's no wonder the United States is as politically divided as it is considering that these services that are so deeply engrained into the lives of everyone clearly favor divisive content like politics that generates engagement. This is a thing that's been known but it's so plainly obvious that it's the case when trying to find information for something specific to your life if you're a minority whose existence is constantly being used as a talking point to signal where you stand in the culture war, and it effectively serves as a reminder to me when I use Google that this is all my existence is to most of society. You might think it sucks trying to find information about a vacuum cleaner or something and getting only shitty SEO'd spam, but you haven't seen anything until you've seen that information about something essential to your existence is nothing more than another piece of chum to trick people into looking at ads.

deltron30302• 5 years ago

I'm wondering how you guys are searching, do you type in short word combinations like 20yrs ago or full sentences and questions? Thing is Google values search intent above anything else right now, and if you don't show clear intent they have to guess, and the selection of search results will be mixed in consequence.

Here's a up to date PDF from Google explaining search intent:

https://static.googleusercontent.com/media/guidelines.raterh...

DoingIsLearning4• 5 years ago

Very useful info but that reads like Apple's "you're using it wrong" press releases.

In my experience Natural Language queries return very poor results except when you are doing a largely popular search.

It is also far more verbose to write: "What is the frequency of the ultrasonic driver in phacoemulsification?"

Instead of: "Phacoemulsification+resonant"

The second query is shorter and gives me a correct answer in ddg whereas google returns irrelevant results mixed in with a long lists of patents.

deltron30302• 5 years ago

Sorry but you're holding it wrong if you don't search intentful enough.

NL queries return poor results if the content creator didn't create it with searchers in mind. Most of these sites also often have a very poor UX.

There's of course a larger incentive to optimize for popular stuff, especially when there's commercial intent (or implied commercial intent by the absence of informational intent).

Bad search results can be opportunities, and many new bloggers focus on this right now which is why they often appear first in search results before the actual experts.

But it would only take experts to improve their content to outrank them because expertise and trust are also ranking factors right behind UX, and if you have both you win. Expertise alone just isn't enough.

projectileboy• 5 years ago

It’s possible that a decent search engine should be considered public infrastructure, just like water and internet and roads and a fire department. If not public, then forming a non-profit may be the way to go. I don’t see how for-profit search doesn’t turn to trash, given sufficient time.

pcurve• 5 years ago

It's not just text search. Bing's Search by Image performs much better than Google's

Retr0id• 5 years ago

And Yandex outperforms them both.

otherotherchris5• 5 years ago

In addition to the site:reddit.com trick, I find prefacing every keyword in intext:keyword prevents most forms of substitution (but disappointingly not stemming) and ensures what you search for is actually on the page.

Any other way of using google lets rankbrain swap your specific engineering jargon for high click-through kardashian news and rap lyrics.

Noumenon723• 5 years ago

I looked up what `intext:foo` does -- it forces the search to have "foo" in the text of the page (not the title or url).

elboru• 5 years ago

I’ve been using HN search a lot more lately. In the past I just used it to find old posts. But now I use it to research topics, the same way I used to use Google.

marstall• 5 years ago

The one thing Google is good at is finding a specific named thing:

- google a company name, it will find the company's website.

- google a historical figure or event, it will bring you to the appropriate wikipedia page.

- google a product, it will show you the amazon link.

Google anything less specific, anything that requires some judgment to discern, and you are dropped you into the SEO bramble of bad information and hostile web design. That Google itself has created.

What a failure for a company with infinite resources, and which has recruited the best minds in computer science.

TechTiki• 5 years ago

I don't believe 100% of the blame lies with Google. It could partly be that there is just a lack of good content on the internet nowadays. Take product reviews for example. You'd have to pay for a web developer, designer, buy a lot of products, hire journalists to test the them and produce content, that all costs a lot. Is it feasible to do this based of a few ads and affiliate links a lot of which will be blocked anyway?

I think what we need is a global micro payment system which enables good content creators to be funded for their work directly.

Traster3• 5 years ago

I’m pretty sure the engineers responsible for Google Search aren’t happy about the quality of results either. I’m wondering if this isn’t really a tech problem but the influence of some suit responsible for quarterly ad revenue increases.

This seems like a pretty irresponsible and ridiculous thing to say. "I think X product is shit, and I'm fairly sure that the engineers working on X would agree and blame product managers" sounds like a valid thing to say if you were at reddit, or dropbox - where it's both true and Seibel should know about it (since he's in a position to know how rubbish they are), but to speculate about one of your competitors in this way is a little... self serving.

Google search actually really reassures me, because Seibel is right, there aren't really any direct network effects, if it got bad I'd move. In fact I did move. I started using duckduckgo when I could, but any time I search for technical issues (something that's likely to result in stackoverflow results or similar) I go back to google. Why? It works.

jccalhoun2• 5 years ago

Agreed. I am really surprised that people aren't calling this out more. It is an example of terrible reasoning to go from "google search is bad" to "it must be bad on purpose so they can sell us stuff" without any actual evidence.

ssiddharth• 5 years ago

I apologise, profusely, if this is bad form to talk about a personal app.

After years of getting steadily deterioring quality search results, and being thoroughly fed up with it, I built an iPhone app (other platforms soon) that removes spammy websites from your Google search results and lets you add your personal, permanent exclusions. Launched it to Reddit which seemed to love the idea: https://searchban.com

freeflight• 5 years ago

This has bothered me for a while now, even DDG is getting increasingly less useful.

Google works if I want to buy something, that's about it. But finding any kind of news or actual information, particularly about incidents in the past, often feels impossible past some Wikipedia article.

At least until narrowing down the date range for the search to escape most of that SEO that just adds whatever you search to make it top of the list.

But even then, on certain topics going back years sometimes yields very weird results, where it feels like there was some kind of purge that only left certain outlets as "valid sources".

Which in practice means the web has become very good at forgetting, as often it's near impossible to rediscover the article for some headline from a decade ago, it's just drowned out by all the SEO if it never made any big waves to begin with.

Wide as an ocean, deep as a puddle, that's what a lot of these results often seem to boil down to, and it's bluntly depressing.

WesolyKubeczek• 5 years ago

My two big beefs with the Google search results right now are these:

1) looks like low-quality linkfarms (like Taboola and that other one) have a big comeback, under the guise of higher-quality content, but Google doesn't give a fuck anymore since 2013 or so when they kept twiddling their algorithm to reduce all the SEO shit;

2) Google started using those low-quality linkfarms and listicles as sources for its "authoritative" onebox answers to your queries ("featured snippets", or "knowledge graph", or how they call it). You look for answers, you get those things front and center as if they are "the" answer. Don't look further. The so-called "deep web" seems no longer to be a thing.

Well, there's also this little problem that for some queries you can easily get a first page of results with one or two organic results and the rest being ads, but that's peanuts compared to the first two.

Imnimo• 5 years ago

I recently tried to find some reviews for a computer I was considering purchasing. A "review" from this website was on the first page:

https://ecomputertips.com/

This is an example of what the reviews read like:

As a desktop manufacturer Dell as an international company has established itself very well in this competitive market of digital gadgets. Check out best Dell desktop computers for specific requirements and beginner’s guide. In this 21st century, a computer has become a very necessary product to everybody’s day to day life.

This is what Google thinks is a front-page relevant search result.

pimterry• 5 years ago

I switched from Google to https://kagi.com recently, I've found them a significant improvement in search result quality so far!

Right now they're free with a waiting list, but the long-term plan is to charge for access - I'm personally hopeful that that's a business model far less likely to incentivize results quality, without falling into this ad trap.

gandalfgreybeer• 5 years ago

I tried that and liked what I was getting. Do you know how much they’ll be charging?

mbarbar• 5 years ago

The FAQ says $10 a month initially.

Freskis• 5 years ago

Can I defend Google here?

What are they supposed to do? The "Internet" as an information resource is dead. All new topical information has moved to walled gardens such as Facebook, TikTok, Instagram, Twitter, Discord, Reddit etc, and mobile apps. The open websites such as Wikipedia, StackOverflow etc dominate everything else. Websites are legacy objects which no-one visits anymore, and they have therefore raced to the bottom to earn scraps of revenue. This is a structural problem that Google cannot fix. Google itself is doomed on the long-term unless it can index new relevant content.

Zamicol2• 5 years ago

There's plenty of great content made by great people. They have the data to solve the problem.

dimgl• 5 years ago

I'm having issues with Google lately too. Every time I look up some piece of code because I want to view documentation related to it, all I get are these example websites that crawl open source pages and index them with SEO optimized queries. Something like hotexamples.com and garbage like that. All of the sudden they're #1 on Google and provide little to no value. It used to be all of the #1 pages were stackoverflow.com or Reddit and the like, which always had really good discussions on these snippets of code.

Edit: in fact, it's gotten so bad that I've stopped using Google for search results and now I go directly to Reddit or Stack Overflow and do my searches there. Which is UNREAL. I don't mind it, but I can't believe how bad it's gotten.

thedailymail• 5 years ago

"Currently, the predominant business model for commercial search engines is advertising. The goals of the advertising business model do not always correspond to providing quality search to users. For example, in our prototype search engine one of the top results for cellular phone is "The Effect of Cellular Phone Use Upon Driver Attention", a study which explains in great detail the distractions and risk associated with conversing on a cell phone while driving. This search result came up first because of its high importance as judged by the PageRank algorithm, an approximation of citation importance on the web [Page, 98]. It is clear that a search engine which was taking money for showing cellular phone ads would have difficulty justifying the page that our system returned to its paying advertisers. For this type of reason and historical experience with other media [Bagdikian 83], we expect that advertising funded search engines will be inherently biased towards the advertisers and away from the needs of the consumers."

The Anatomy of a Large-Scale Hypertextual Web Search Engine, Sergey Brin and Lawrence Page. http://infolab.stanford.edu/pub/papers/google.pdf

redisman3• 5 years ago

Beautiful

toomim• 5 years ago

Wow. This should be the top comment.

yawaworht1978• 5 years ago

I agree, the results are too heavily influenced by domains like reddit, Pinterest, Quora.

The next thing I have realized, if I am looking for something about a topic or a person and there are current events heavily in the media on the person or the issue, it's almost impossible to get something useful or even related.

Stack overflow results are sometimes replaced by less useful GitHub links.

Then , job search results are pretty bad too , same for shopping items.

It seems like media outlets and the big traffic domains get preferential treatment.

And this is just for English, God knows how bad this will be in other languages.

emptybottle• 5 years ago

If you haven't switched your default search to duck duck go yet I highly recommend doing so.

As a user of DDG for years now rarely does searching with !g give better results.

At this point its the opposite. The DDG first page results aren’t all ads above the fold.

oblib• 5 years ago

I've noticed this too. In fact, some of their results are pretty close to worthless. Last week I posted a link here to a demo I made for creating and using "blobs" with PouchDB. I titled it "PouchDB/CouchDB Save/Update/Load Image Blob Demo".

After I posted it I monitored my web logs for a bit and watched both Google and DuckDuckGo hit the page.

3 days later I did a search for "PouchDB Image Blob Demo" on both Google and DuckDuckGo.

Google had almost nothing at all for that search and my page wasn't on any of the results they offered. DuckDuckGo had my post at #1 for that search.

lifeplusplus2• 5 years ago

100% I end up prefixing my searches with Reddit it.. or stackoverflow. It wasn't the case even just 4 years ago.. now if if top results aren't it chances are that rest is even more completely off mark

firebaze2• 5 years ago

The most advanced AI staff, their algorithms beating even go champions, still being worse in search than ever before. Beaten by bing.com, yandex.com and others, at least in my humble opinion.

Something must be wrong, but I don't know what.

ilamont• 5 years ago

This was the nail in the coffin for me:

When did Neil Armstrong set foot on Mars? (https://news.ycombinator.com/item?id=28224730) (October 2021)

MiddleEndian4• 5 years ago

Months later and the live search still puts "July 20, 1969" at the top.

Noumenon723• 5 years ago

To be fair, I as a human didn't notice it said "Mars" either and thought the answer was correct.

anotheraccount9• 5 years ago

It's time for a(nother) revolution with search engines. The web is, more than ever, filled with affiliate links, retarded ads, and crappy sites that exist for the sole purpose of redirecting traffic. Most of us browse less 2% of the web. I miss Archie and Veronica. I'm probably complaining too much and totally out there, but I miss discovering insane intellectuals, incredibly unique and valuable content, rebellious nerds, everyday. (Yes, I'm asking a lot). There's many brilliant people writing high quality content online, but's too diluted, hidden, forgotten, invisible, lost. Sometimes browsing feels like changing channels on a tv. I feel trapped in a rotten loop. I need a search engine that will rock my world. Show me the real stuff - I know it's there. Rant over.

rc_mob3• 5 years ago

I just want 2002 version of google back lol. it was perfect

busymom0• 5 years ago

I have observed this for the past 3 years now. For example if I search for "reddit best soup pots" and set the date filter to be within last year, it gives me results which are 7 to 9 years old. This used to work perfectly fine 3 years ago.

amarento• 5 years ago

helllloooo ... does anyone from Google Search engineering, product management, and/or leadership have any comments on this ... if this is true, at this rate Google Search, the foundation of the whole Google / Alphabet enterprise, faces the existential threat of becoming irrelevant in the world of search

aronpye2• 5 years ago

How are these sites created? Are they hand crafted or automatically generated with something like GPT?

From what I’ve seen, a lot of them just seem to copy-paste content from each other and sites like stack-overflow. I’m just curious whether a human does this or a machine programmed to game SEO.

lettergram• 5 years ago

Interestingly enough, I've been frustrated by this for years

https://austingwalters.com/is-search-solved/

I used to manage hnprofile.com which utilized a patent I wrote to target this exact problem. Effectively, Google optimizes for ads (as pointed out) and optimizes to ensure you have to click multiple links. In reality, we want to answer peoples question(s) right off the bat, i.e. no ads.

How do you make money then?

Well, that's why I created this: https://insideropinion.com (or https://metacortex.me/)

I think the only way to make profit off of it is by targeting corporations, where their revenue comes from maximizing productivity. I think it's possible to create a paid service ($5/month) for good search, but you'd still likely have to target companies.

At least that's the best I could come up with.

Cwizard• 5 years ago

Something I have noticed lately is that when searching for technical information I often get result near the top of my search that are crappy skins of StackOverflow, usually with many more ads. Has anyone else noticed this too?

effable• 5 years ago

Yes, I have definitely experienced this too. I also sometimes get results from websites that seem to be reskins of github (usually issues and pull requests).

zelon88• 5 years ago

You mean to tell me that when you use Google design paradigms, to implement Google promoted "best practices" in a way that scores highly on Googles own test suite and build it on Google promoted Javascript-by-the-pound libraries and then strap it with Google analytics and ads, promote it with Google ads, and then host it on Google cloud platform that you're actually building a dirtier, shittier web?

Well paint me surprised.

xenihn• 5 years ago

This quote from a Wikipedia article about a Heinlein book is apt for this. It makes me think that what has happened here is not only inevitable for any disruptive tech company that grows large enough, but also any other sort of human collective in general:

"This theme is echoed elsewhere in Heinlein's works – that real liberty is to be found among the pioneer societies out along the advancing frontier, but the regimentation and legalism that follow bring restraints that chafe true individualists."

aantix• 5 years ago

What does it take these days to roll our own search engine?

I’m working on a new kind of search - I’d like to create my own independent index. Bing is fine, but restrictive. Gigablast is ok.

https://search-new.herokuapp.com/

Looks like the Common Crawl .WET files are about 10 TB (https://commoncrawl.org/2021/11/october-2021-crawl-archive-n... ).

Typesense recommends 3X the amount of RAM to hold the indexes.

30TB of RAM. Each TB server, what, $5k?

I'm sure some pages can be reduced. Worst case, $150,000 for server costs?

ur-whale• 5 years ago

From the Twitter thread:

I’m pretty sure the engineers responsible for Google Search aren’t happy about the quality of results either.

Strongly disagree.

I'm pretty sure the kind of drive and passion that led to the very high quality of Google search back in the day is long gone.

At this point, people do not join Google for the technical challenge or the reputation and/or ethos of the company, but for the fat bonuses and RSU grants.

evouga2• 5 years ago

It's not just Google---the World Wide Web itself is rapidly becoming a defunct protocol, the culmination of a decades-long shift in the Internet's center of mass away from browsers and towards centralized and commercialized apps---from personal web pages to LinkedIn/GitHub/Twitter handles, from the ubiquitous WordPress blogs to YouTube videos and Medium posts, and from forums to Tweets and subreddits.

The useful information on the Web is now concentrated in a few places---Reddit, Quora, Stack Exchange, Wikipedia, etc. When I want reasonable search results, I search there instead of the Web as a whole. (And even many of these services are in the late stages of the Silicon Valley life cycle---desperate monetization and engagement-increasing gimmicks---with uncertain futures.)

Wikipedia, bless its heart, lurches on as a cathedral to early Web's dream of information democratization. It stands as a wonder of the ancient world: incomplete, built from technology few now understand, and reflecting values and priorities that no longer quite align with contemporary culture. But it persists thanks to its inertia and the undeniable sense of awe it invokes even today.

topspin2• 5 years ago

Discord is another tarpit into which a lot of otherwise indexable content is sinking.

visarga6• 5 years ago

And Slack. Can't ever find what I want in it.

fallous2• 5 years ago

Wikipedia is also suffering under the issues of "moderation." The original web was based on the idea of "throw your crap up, if people like it they do if not they don't." Evaluation both the content and the credibility of content on the net was up to the viewer. Adding moderators apart from the original authors and viewers, especially when those moderators are at an integration level (be it wikipedia, twitter, google, etc) is in direct opposition to that original ethos.

YPPH• 5 years ago

Actually, Wikipedia has similar issues to Google SEO spam, in the sense you now have people who pay marketing firms for articles to be written on themselves/their company. These articles, unsurprisingly, often infringe core policies like maintaining a neutral point of view.

With respect, I don't think the problem is as bad as you suggest. Some pretty awful articles have survived deletion processes so long as they are on a notable topic.

xiphias25• 5 years ago

I read this quite often on HN, but my experience is quite different: English Wikipedia pages are amazing for learning about some scientific or historical thing or when I want to understand something. Even with political things it tries to describe the controversies without being too judgemental (which is really hard to balance out).

netcan• 5 years ago

Wikipedia also suffers from the "try to be good and they'll hang you" problem.

Whether it's moderation, deletionism, or annual budgets... Wikipedia always seems to have detractors enough to make exon-mobile shudder.

feupan• 5 years ago

built from technology few now understand,

Now you’re just being dramatic. I’d review the whole comment and de-dramatize it.

Yes the web has changed and most Forums are dead, but it doesn’t mean change is not possible. Blogs are still a thing, even if you get to them through Twitter.

The issue stems from Google rewarding junk and people wanting to monetize their content. Couple the two and you have lengthy BS lost in ads, while short clean content is lost on page 59 – but it’s there.

egKYzyXeIL• 5 years ago

Maybe it's just me and my English comprehension, but way too often I am left thinking comments on HN have become a sport of dramatization and usage of thesaurus

aluminussoma• 5 years ago

It got a chuckle out of me. I went through the Wikimedia source to track down a bug and I can agree with the sentiment.

I guess most technologies are like that. Once it is written, once the major bugs are caught, few revisit the code anymore.

evouga2• 5 years ago

Back in the day I both edited Wikipedia and helped maintain a MediaWiki installation. I found learning wiki markup (and the subtleties of the template engine in particular) quite challenging on the frontend, and the backend was even worse.

The inaccessibility of the software is (was?) widely acknowledged as a significant barrier to new contributors by the project itself. I didn’t think this was especially dramatic or controversial.

mikewarot2• 5 years ago

Wikipedia went down quite a few notches in my world view a few days ago, when I went to find out how old Lex Fridman was from his wikipedia page

https://en.wikipedia.org/wiki/LexFridman

Only to find he's not notable enough for an entry. There isn't an article in a "reliable" source about him, so he doesn't exist as far as the club that now manages Wikipedia is concerned. I wasted about an hour seeing if there was something applicable I could add, but to no avail, as Google failed me.

BridgetJones• 5 years ago

I'm curious if the fault is Google's, Wikipedia's or "the Web's"?

edgyquant2• 5 years ago

This is hilariously pedantic. For one thing it would be https://en.wikipedia.org/wiki/Lex_Fridman and this person isn’t all that popular. If you think he deserves a page create one

dennis-tra• 5 years ago

What a coincidence, just today I had a conversation about the decreased quality of Google’s search results. Glad, I’m not alone.

I’ll give you.com a full weeks trial as it wasn’t mentioned that often in the comments yet.

Their CEO is following the twitter thread [0] and comments here [1] but is probably hesitant to advertise it here on HN.

So, I’m doing it now as I have high expectations. I’m not affiliated in any way.

[0] https://twitter.com/richardsocher/status/1477748601539411971...

[1] https://news.ycombinator.com/threads?id=richardsocher#293994...

phkahler• 5 years ago

> This is why no software incumbent is truly protected from startup disruption

Not quite. It's why publicly traded companies are not protected, they can never make enough profit. At least not in today's world where the whole market looks like a scam.

com2kid2• 5 years ago

Someone could make a fortune launching a search engine running Google's algorithms circa 2010!

Actually, thinking about it, Google has always been fighting low quality results from spammers and SEO. I remember 2017-2019 there were multiple topics that you just couldn't research on Google because the SEO blog spam was so overwhelming.

I can confirm that recipes are a mixed bag, but some team at Google is working hard to try and make them good, it is just a really difficult fight. It doesn't help that the number of recipe sites keeps exploding.

Oh by the way, if you want good recipes, just pay for a Cook's Illustrated subscription. There. Done. Sadly NYT Cooking has started adding some really low quality recipes that honestly no one should be making.

bonyt• 5 years ago

One category I’ve noticed this kind of thing in is calorie counts. I’ll search for a product that should have an official nutrition page from the brand’s website, and get pages and pages of websites that just seem to regurgitate from some large database.

I use MyFitnessPal to log calories, and it already has one of these large databases. I’m usually searching online to validate it against another source, so this is pretty unhelpful.

onion2k2• 5 years ago

I've thought about this a lot over the past few years, and I've come to the slightly sad conclusion that Google isn't to blame. It's just that there isn't any good quality information on the web anymore. Google does its best, but it's working with crappy data. No one wants to spend time and effort making a great website about <insert literally any topic here> unless they're in it for the money. That means Google will only ever return blogspam, affiliate websites, and SEO-optimized ecommerce websites for practically any search now. This is in contrast to the good old days when people made websites for fun and searches actually found the high quality content that was out there. These days searches don't find quality content because, on the whole, it doesn't exist.

This isn't universally true of course. There is some good content. But it's never what you're searching for; it's only good when you stumble across it, or you find a link on HN/Reddit/etc. It's just interesting rather than specifically good or useful.

To an extent Google is to blame because AdSense and DoubleClick drove the shift from people publishing what they love to people publishing for dollars, but, and this is somewhat cynical I know, I genuinely wonder if we're actually on the brink of realising the web as a publishing platform just isn't that great.

overkill28• 5 years ago

Yeah much of the quality content for recipes and product reviews have moved to video, and two of the most popular platforms to host it on (Instagram and TikTok) are walled gardens.

Rastonbury2• 5 years ago

Really? I am immediately skeptical of random influencer recommendations especially on social media, its slightly better on youtube where respected creators/reviewers are transparent about sponsorships to maintain their credibility

christophilus• 5 years ago

There is good content, it just never shows up in search results. So yes, I blame Google. Their ability to ban obvious spam / copycat content is laughably bad.

o10449366• 5 years ago

I disagree with the notion that good quality content just doesn't exist anymore. Quality content will always exist and more quality content exists now on the internet than ever before now that it's become more accessible to more people than ever, but Google, Facebook, and Amazon have made this content more difficult to discover because their algorithms reward gamification with profit, incentivizing the production of high-volume and low-quality but highly-optimized content.

People passionate about sharing and learning will always be driven to produce quality content, even if they don't have an audience. The issue is that Google will never discover these people because they're only interested in discovering the best marketers on the internet.

onion2k2• 5 years ago

People passionate about sharing and learning will always be driven to produce quality content, even if they don't have an audience.

Those people moved to content websites like YouTube and Udemy where they can cash in on what they do. They don't make websites any more, so Google doesn't find links to them.

visarga6• 5 years ago

It's true that we're drowning in shit, but at the same time there has never been more useful content online. Google is just incapable of ranking it, or it doesn't suit their financial interests.

EscargotCult• 5 years ago

I've thought about this a lot over the past few years, and I've come to the slightly sad conclusion that Google isn't to blame.

I don't know about that, at least in the realm of programming-related searches. It's ridiculously frustrating when I'm searching for a standard library function and the top 3 results are geeksforgeeks, w3schools, and tutorialspoint, while the canonical documentation for the language is only 4th or 5th in the results.

vladf• 5 years ago

Funny, I was just complaining about the exact same phenomenon (complaining with a fix!) a few days ago: https://twitter.com/FeinbergVlad/status/1475148590423023619

impalallama2• 5 years ago

The fact that MDN Docs are never the first results when searching basic for most basic web stuff is very disappointing.

skinkestek5• 5 years ago

I've thought about this a lot over the past few years, and I've come to the slightly sad conclusion that Google isn't to blame. It's just that there isn't any good quality information on the web anymore.

Have you tried marginalia and kagi?

If not, come back after you have tried.

Until I tried those I could have thought you were possibly right.

After testing those I am again confident it is just Google and DuckDuckGo.com who fails, which has been my position since a decade or so ago.

unnouinceput2• 5 years ago

The comments on both Twitter thread and here on HN goes something like "better have your own communities on Reddit and StackOverflow for your advanced queries". Sooo, we go back to Yahoo's grouping/directories that was the norm in 90's, eh? And Google bested that by going with their unique search algorithm in early 2000's. Hence next is going to be a reinvention of the wheel by DuckDuckGo I suppose.

jeffbee• 5 years ago

DDG has literally zero search technology.

unnouinceput2• 5 years ago

I don't know about that. I use Firefox with DDG as default search engine and I go to Google only in case of searching for local stuff (as in my country's local stuff). For everything else DDG pretty much covers my needs.

jlarocco• 5 years ago

I'm surprised anybody needs a random guy on Twitter to tell them this...

dpweb• 5 years ago

G could just offer an ad free tier included with Google One or something.

throaway6942• 5 years ago

for opinions like product reviews or entertainment I usually append my search query with 'reddit'

commandlinefan• 5 years ago

I’m pretty sure the engineers responsible for Google Search

I have the distinct impression that engineers haven't been responsible for Google Search for a long, long time, having been supplanted by biased activists.

xibalba• 5 years ago

removed

pxmpxm6• 5 years ago

What's even the definition of "inclusive language"? I thought the whole idea behind certain groups embracing post-modernism lately was to avoid inconvenient things like reality; meaning is meaningless.

jccalhoun2• 5 years ago

Google is now openly boosting/penalizing pages based on "inclusiveness"[1] in language

I just read the transcript of that podcast found here: file:///C:/Users/jccal/AppData/Local/Temp/Search_Off_the_Record_-_27th_episode_1.pdf and that is NOT what they claim they are doing.

It seems like what they are doing is basically using synonyms. From the transcript: "So if you search for something like fireman, it would automatically include the different versions, including websites that mention firewoman or fireperson." (most of their examples are French or other languages that have more genderded forms of words than english)

They also say, "we are not in a position to tell what people-- how people should write."

the__alchemist2• 5 years ago

Unfortunately, there's no superior alternative. Anecdotally, Google produces substantially more relevant results (For things I commonly search for) than Bing and DDG.

james-redwood• 5 years ago

Try Kagi or Neeva. Both have far better results than google and support bangs as well as extensive customisation.

the__alchemist2• 5 years ago

Kagi seems to be limited access, and Neeva spammed me with registration popups almost immediately. I'll investigate both more, but this isn't a promising start.

crazypython• 5 years ago

Brave Search and Bing wrappers like DuckDuckGo and EntireWeb as well as niche search engines like deephn.org and twitter.com are better

63• 5 years ago

I'd love to know why Google has started ignoring quoted terms and treating them like normal words in a query. I put it in quotes because I wanted that exact phrase, not something that sounds similar. If that exact phrase doesn't exist, tell me.

DoingIsLearning4• 5 years ago

I have repeated this comment before and it's wild speculation but I try to answer. For me google's results were good up to 2016 then the results started to become worse and power user features like bolean operators and keywords ("site:", "domain:", etc.) started to get ignored.

I map this decline to the exit (with some sex scandals mixed in) of Ranking VP Amit Singhal and the start of John Giannandrea leading R&D. From what I understand from the grapevine Singhal was uber conservative about how anything AI would scale in the ranking algos whereas Giannandrea was pretty much a 'let's AI all the things' kind of guy.

This aligns with the start of forced 'natural language' queries. After this roll out you now have to second guess how AI expects a query in order to get half decent results out of google.

As an example, in the early 2000's you could just type: "phacoemulsification+resonant"

Nowadays the expectation from googles AI is to query along the lines of: "What is the frequency of the ultrasonic driver in phacoemulsification?"

(BTW the first query still gives me a correct answer in ddg.)

In addition to the AI galore, google dropped all the power-user feature keywords (as in they are still there but we will politely ignore them when convenient),

Also from forum discussion the GOOG employees feedback was always that the bolean opearators were underused (probably only a few millions of users?!) so it got removed "but you can use quotes", because it is so much more convenient.

Google search has been slowly adding just too much friction. Now I just use ddg and get decent results 85% of time with zero friction/frustration.

For my use cases the only scenario where google's AI shines is in 'Google Shopping' product/price search, for everything else it truly feels like the right tech for the wrong problem.

cjlovett• 5 years ago

It's amazing to me it's 2022 and Google search results are still so useless. I'm finding myself use duckduckgo a lot more these days.

alfiedotwtf• 5 years ago

As long as Pintrest keeps coming up in the top 5 results, the title holds

fnord773• 5 years ago

protip: add "forum" to your search. you'll get hits from actual forums and you'll bypass a lot of seo crap

londons_explore2• 5 years ago

Is there good evidence that Google isn't doing a good job of finding content anymore, or that good content doesn't exist on the public internet anymore?

Google is only doing a bad job if what you're looking for exists but can't be found. I have a feeling that for many searches, what you're looking for simply doesn't exist anymore.

cmurf• 5 years ago

Not news, it's been getting shitty for awhile to the point most of my searching uses "" to force those keywords to show up. And now I'm regularly getting results in which the page doesn't contain required keywords. Super aggravating.

AviationAtom• 5 years ago

uBlock Origin is a must when clicking on some first page Google results how, as well as a good pause and study of the results. The unscrupulous ad baiters have gotten real good at Google SEO and content scraping. I'd guess they do well for themselves, as the practice has picked up significantly in recent times. It reminds me of part of the reason Google stole Yahoo's marketshare so rapidly: if you deliver useless results too often then it's a waste of my time to use you. Perhaps the time has come for a new breed of search engine, to disrupt the search engine game, which it seems Google shifted their focus from long ago.

hintymad• 5 years ago

I grew up with the amazing stories about Google engineers, and I wish Google will not repeat the history of companies declining gradually then collapsing suddenly, as it is such a revolutionary company not just for its product but also for its amazing contribution to the technology. The sad thing is that I'm seeing more and more cracks in this great company.

visarga6• 5 years ago

Google, why are you slacking off search and also the Google Assistant? It feels like a rerun of the 2003-2010 period in Microsoft.

Web search is getting worse, people complain, problems persist. I thought your mission was to organise the information and make it useful.

And the Assistant is just as dumb as years ago, while NLP has been progressing leaps and bounds in the last 2-3 years. Where's the progress? Are you aware speech is going to dominate direct web search?

agencies• 5 years ago

Who has concrete steps to make this better? Seems like one or two people are making their own engines, but moving the needle is going to take a lot more than that...

yashap• 5 years ago

Yeah, the results are absolute SEO garbage far too frequently. However, I'm not sure if this is Google "losing" to SEO ppl, or short-sighted greedy behaviour by Google ads ppl. It certainly could be the greedy case - SEO garbage pages tend to be packed full of Google Display ads, so Google gets paid by advertisers whenever you load them. If they are favouring pages with Google Display ads (or favouring domains that spend a lot of Google Search ads in "organic" results), then that'd hurt the results a lot.

Example of terrible SEO results - I saw an interesting magic trick on Reddit, and someone in the comments mentioned they did it using "Key BDM Scissors". I tried searching all sorts of different things along the lines of "how do Key BDM Scissors work", but literally every single result was just online stores selling the scissors, with the word-for-word exact same blurb on each page. No matter how I tweaked the query, the results were identical, and in no way explained how the trick scissors worked. Useless.

mwaitjmp• 5 years ago

Searching for content which you know is on GitHub is the biggest issue for me. Now rather than returning results for code from GitHub, you instead get results from sites which copy content from GitHub.

It’s actually useful they perform this or else some of that content would never appear in Google’s search results.

Volker_W29• 5 years ago

Can you guys give me some good examples of sites that are "SEO crap" or google searches that do not return anything useful? I want to do some experients if I can write a program that takes a search query, hands it to google, takes googles result and reorders them a bit.

futuretaint2• 5 years ago

the free internet was cancelled w/ Richard Stallman I guess. This is what we get. what a great future we computer folks have provided to our next generation.

lbrito• 5 years ago

Search results are decent for most technical stuff, but anything consumer product-related is abysmal. I always seem forced to go to YouTube for that.

When I'm searching for "best short throw projector", most of all the results on Google are SEO spam. The same search on YouTube has a lot of low quality YouTuber garbage as well, but there are at least a couple of high quality results where the host tests input lag, fan loudness etc of several projectors. This is the kind of content that used to exist in sites (I remember Tom's Hardware from the early 2000s), but now seems to be hidden deep in YouTube only.

moonshotideas• 5 years ago

It seems like most of the comments seem to be on one of two sides. 1. Learn to live with this, and you’ve helped creat this problem 2. This is terrible but some other non related point

It seems to be that the original only grail algorithm -page rank, that made google the superstar it is, can be hacked in a way that saturates the internet with low quality search results.

The obvious question seems to be what algorithm could be used to identify/differentiate low quality websites from high quality sites

belfalas• 5 years ago

The more I think about this, the more it looks like classic short term thinking. Juice ad revenue in the short run. Open the door to complete disruption in the long run…

This is the key insight from the thread. Publicly traded companies get pulled into the short term thinking cycle and this is the inevitable outcome.

calltrak2• 5 years ago

Here is a list of google search alternatives https://fabform.io/a/alternative-search-engines

gmi01• 5 years ago

I find myself adding the word “Reddit” after certain type of keyword searches so I can get real people opinions and experience on things.

bamboozled• 5 years ago

On a side note, I went to use Google places API yesterday and the offering is trash. They get reviews for free (including my own) then expect me to pay through the nose to access that data and guess what, if you want to cache any of the results for performance and reliability reasons you’re violating their terms of service because that would impact their bottom line.

Disgusting.

dilawar• 5 years ago

For the last 6 months or so, I started maintaining bookmarks because of frustration with the search results. Occasionally, a useful search result is found on 2 or 3rd page.

I also installed https://iorate.github.io/ublacklist/ recently to block some annoying sites from results such as geeksforgeek polluting C++ related searches. I am uncertain if this is inevitable because I am increasing interested in higher-quality content, while most content on popular (SEO optimized?) sites is definitely not the best.

I am still happy with Google Scholar results, though I have no benchmark to compare against. ReserachGate?

tiberriver256• 5 years ago

Could this be a side effect of web developers moving away from traditional websites to single page web apps? Single page apps are notoriously difficult to make SEO friendly but right now they are the hotness. A 'crappy' WordPress site will be much easier to get ranked on Google than a multi million dollar angular site.

bromuro2• 5 years ago

I am using Google to look for programming documentation or solutions to particular problems. Top results are very often unrelated spam from Medium, clones of SO, SEO website against npm, etc

It’s quite hard to make specific searches - it insists to show me generic results, either by “did you mean?”, or excluding the keyword I explicitly added, or because it thinks I am looking for something more common.

If there are no results they should just tell me - instead of trying to keep me “engaged” with their spam.

Volker_W29• 5 years ago

Maybe there is a lot of anti-Google lies in this comment section.

I could not reproduce their problems when I googled the same string.

csee• 5 years ago

Part of the degradation was when they got rid of URLs. I had a really good bullshit detector function in my head. The URL would go into it, and an intuition would pop out that it was a bullshit link. Now I can't do that.

summerlight• 5 years ago

Beside of having more and more ads in the search results (which I think mostly driven by the pandemic because of significant revenue loss in 2020 Q2?), I think the web itself is the most significant offender of this problem. Not only those annoying SEO (which makes the problem exponentially harder), now more and more quality contents are going beyond unsearchable walled gardens. Basically almost all web sites from mid-sized tech companies now require some sort of sign-in by default and if you try to crawl it at scale, you'll get banned. Perhaps Twitter is the only significant exception to this trend.

technotarek• 5 years ago

To all those hating on affiliate marketing and how google is ruining the internet as a place for reliable product research, I propose a different take. Affiliate marketing is actually a way to fund product research and testing. The problem is those that currently pay out are too few. Imagine if all products/sellers provided a potential payout equally on par with one another? Then you could help protect against the bias that leads to only Amazon and Walmart stocked reviews. To boot, you create incentives to shop from a wider scope of vendors and marketplaces.

We do affiliate marketing on ATTIC.city that we believe is honest and fair. We let every product on, from almost any vendor (that we can technically support), regardless of whether we have a commission agreement. Our search results always give the best match that we can for your query. Period.

mhb• 5 years ago

Any way web of trust can fix this?

SMAAART• 5 years ago

oh that's interesting.

let's all remember Clayton M. Christensen's lesson: a market can only d=be disrupted once it's overserved.

Guess search/google is due for disruption. DuckDuckGo is looking good, but maybe it's going to be a second-mover (second after DuckDuckGo).

blondie9x• 5 years ago

Kind of scary when you realize there are only really two search engines in the most of the world. Google and Bing. Google and Microsoft. The world deserves more than this. DDG, Ecosia, Yahoo, etc are just Bing behind the scenes.

calltrak2• 5 years ago

Here is a list of alternative search engines

https://fabform.io/a/alternative-search-engines

servytor• 5 years ago

The future of search isn't better search results - it's a customized solution to you and what you are looking for at that moment. I imagine a future where Google returns an AI interactive summary query with sources.

But at the same time I always thought of how odd it is that Google competes in an area where there is no user lock-in whatsoever - I mean search should be one of the most easy areas to disrupt - with almost no competitors.

gremlinsinc2• 5 years ago

Ironically, I think reddit's search sucks, but the knowledge of reddit is pretty decent, so searching google and adding reddit gets some of the better recommendations... at least you skip the shitty ad sites.

gremlinsinc2• 5 years ago

This is actually one place I think blockchain could make sense...

Use people's servers as a sort of botnet to process search results, the more you process the more you earn, you'd need maybe storage nodes for indexes, processing nodes for ai/ml categorization of results, and then maybe people can also earn coinage for ranking results they do get, and that way the shitty sites could be more likely filtered out.

To advertise, costs coins which allows advertisers to basically buy ads not from the SE itself but from users via a marketplace.

r0m4n0• 5 years ago

I’m slightly biased (as a googler) but why is this person’s tweet newsworthy? It doesn’t really contain any revelations or specific examples

Transisto• 5 years ago

At least 3 times in the last week I've noticed that trying to search for information that is not the government narrative about covid vaccine leads to the crappiest piece of garbage having an attempt at discrediting whatever I'm searching on.

Google / YouTube suggestion engine is now evil and corrupt to the bone.

thesquib• 5 years ago

I have to agree. Lately (maybe year or so) it seems that researching anything that isn't clearly linked to some kind of product or service is difficult on Google.

It occurs to me that it could partly be that the "free and open" internet seems to be dying. For example, Discord or Slack are often used to replace message boards but that means they are no longer indexed and searchable. There are many examples like this right?

The old free hosted sites are dying off or provide no monetary value to Google. So even if they still exist, why bother indexing them.

This has nothing to back it up of course, just musings.

zbtaylor1• 5 years ago

My first web job was building websites for a rinky-dink SEO company. My boss had zero qualms churning out trash content while trying to game the system. How can Google fight an infinite sea of these operations? I'm not sure who's to blame at this point.

Like many HN users, I assume, my method for finding useful info online has always been to find a forum. You can't monetize a forum post without it being ignored or discouraged by the community so until that changes forums will always provide the most consistently helpful content on the net.

Obviously these companies are making enough money off the segment of the population that does not behave this way to make it worth their while. My question is how big is this segment and will it slowly dissolve as older generations pass and more grow up with the internet? Or will there always be enough people susceptible to this spam that the internet will never get better?

fallous2• 5 years ago

How shocking. An advertising company who uses search as a venue for advertising does not produce the highest quality of search results for those who search.

Stop thinking of google/alphabet/insertnewbrandnamehere as a search company. They attempted to make money from the search engine in the 1990s when they couldn't find customers willing to pay for that technology and the company chose to make money from ads. Your company is defined by how you make money, not by user perception or press releases. Google is an ad company because they make money off of advertising. Goodies they give you that leads to better targeting of advertising for them is merely good ad business, stop thinking it's for any other purpose.

seventytwo3• 5 years ago

100%.

I’ve been noticing Google returning lower quality results consistently for at least a year.

My guess is that the results are based on relevant info, but also on which pages can serve up the most ads at the same time. Because the internet is heavily influenced by search algos, over time, this decision feeds back on itself and results in search results being spammier, less content rich, and less accurate.

Lamad123• 5 years ago

Sometimes I search a big chunk of lyrics from some fairly popular song that I hear in several radio stations and cannot find it... This wasn't the case a few years ago.

whiddershins• 5 years ago

A challenge I see for anyone trying to disrupt this space is that in principle Google could fix this at any moment.

So a startup would still need some secret sauce, otherwise they might gain a toehold just to get absolutely wrecked by Google tweaking their algorithms a small amount and regaining dominance.

haldujai• 5 years ago

I mean isn’t the issue at hand a by product of how massive the internet is now and Google’s relative impartiality?

In a truly inclusive and expansive index isn’t it a bit of cat and mouse with SEO? Presumably the only way to get rid of them would be for Google to basically start curating the web. It’s an interesting thought but a bit unsettling at first. A secondary/premium product line or competitor seems more palatable.

kristopolous2• 5 years ago

I searched this entire thread for the word "fidelity" and got 0 results. The problem is the sophistication of the web has lost cadence with the coarseness of the tools. There is no way to get to there from here. What search means and how it is interfaced has to change.

TroyMacLure• 5 years ago

Search for "more evil than the devil"... Reference to some rich entreprener...

We used to say that what is brought on the Web stay on the Web... Not anymore.

Marketing ruinned everything, as usual.

Ostrogodsky• 5 years ago

Google no longer producing high quality search results in significant categories
Amazon no longer producing high quality products in significant categories
Netflix no longer producing high quality movies search results in significant categories
Facebook no longer producing high quality anything in significant categories

nutate• 5 years ago

Google search is shot. They need to revert the jillion microservices to pre-covid versions and just let go of whatever they've done. The changes have surely increased some key performance indicators but they've also made their service look like its always April 1st.

rmason• 5 years ago

PG did a tweet stream on Google's vulnerability and how a startup might attack it.

https://twitter.com/paulg/status/1477760548787920901

thunkshift1• 5 years ago

Isnt that the yc guy? This seems to me like a call for entrepreneurial people to come up with google alternatives just so that YC can dip their fingers in it while doing jack.

randomperson_24• 5 years ago

I think the problem is that we all expect to get perfect results. We want automated systems to identify and remove unimportant useless things. But how will those systems get to know that?

We can't build another AI black box to try and do that. There is a option in search to report spam or improve search results.

Out of our laziness and expecting getting everything perfect, sometimes collectively working and reporting bad content-copying websites, etc. would be collectively useful for all.

Search had gotten so good that we probably expected to get perfect results everytime. Untill bad guys started to game it. I beleive it's more of cat and mouse game, and where all users collectively should help and mark as spam.

yourapostasy10• 5 years ago

...collectively working and reporting bad content-copying websites, etc. would be collectively useful for all.

How would you combat commercial actors weaponizing this reporting to perform takedowns of legitimate competitor websites via distributed bot reporting attack (DRoC, distributed reporting of content attack)?

de6u99er• 5 years ago

Google search results are actually horrible. My partner and me were searcbing today for the annual budged of public media in Poland. A couple of years ago the top result would have been the correct answer. Today we only got crap and gave up after 5 pages (sear result pages) clicking through ad infested links.

Two days ago I searched for a mission description of a game because my partner was talking to me and I only got half of the mission description. Top results were sites which copy headlines from forums and display tons of advertisements. This made me furious and I wouldn't be surprised if Google is by design ranking search results from certain categories based on advertisement real estate. The more Google-ads the more prominent the search result.

I make Pichai responsible for all the spam. He is, in my opinion, actually an idiot who doesn't understand that his strategy will be the downfall of Google as THE search engine. I really miss Page and Brin and their "Don't Be Evil" culture.

Nginx487• 5 years ago

Google is producing predatory monopolistic practices, blocking users and deplatforming businesses amidst pandemics without any sane reason, destroying businesses and ruining lives.

stoundmire• 5 years ago

It's been bothering me for a while. Whenever I search anything programming related, low quality websites (like w3schools and geeksforgeeks) always rank on top. I have to write a script that attach

-site:w3schools.com -site:geeksforgeeks.org -site:programiz.com -site:realpython.com -site:tutorialspoint.com

to every search I make, in order to avoid all these BS and get to the official sites.

But bad news is, that list keeps growing and it's impossible to filter out every single one of them.

causi• 5 years ago

Google is now regularly ignoring search constraints like the - operator to remove results containing a term and "" marks to only give results with a certain term. This is quite infuriating when you're searching for something that sounds like something much more common but isn't.

dreamcompiler• 5 years ago

One thing that would help immensely is a programmatic API for search. That way anybody could write a program to contact a REST search server and get back 100 results, and then filter out sites that were on one's personal blacklist, and also check the remaining results to see that they did in fact contain the search terms requested.

If you try this with Google you'll immediately be captcha'd to hell and back. You can do it with a browser extension (and many of these exist) but you're always at the mercy of the browser maker (often Google itself) who can arbitrarily decide break your extension.

Programmatic access to the web was exactly the premise of the so-called "semantic web" from a few years back but it never caught on. We probably need a government-funded search spider and database that is mandated by law to allow REST access and which can never be commercialized. Of course any attempt to build such a thing would be lobbied into the ground by Google, but I can dream.

equasar• 5 years ago

Google will block your API/site right away

pugets• 5 years ago

Google results change frequently, even when they shouldn’t. Searching for “shrimp recipes” gives me mostly food blog articles that are from the years 2020 or 2021. Why would I want recent recipes over tried and true recipes? Where did the good recipes from 2013 go?

Shrimp recipes may not be a “significant category” but this I think highlights one of Google’s search issues. It’s also how they make money. If there were no revolving door of promoted content, then how would today’s advertisers ever reach you? It’s a balancing act to maximize profits while minimizing damage, and they aren’t doing a good job with balancing.

_hcuq• 5 years ago

Time for Noogle - a brand new search engine. Some money to be made there…

z0xz0xz0x• 5 years ago

He wonders what a paid version of Google search would look like. I don't think most people would be able to afford such a service and it'd be reserved mostly for people with lots of disposable income. However at that point, you're losing your most valuable ad clicks. From what I understand a single click on a Crypto.com ad can cost $40-$60 dollars. I don't see it being a sustainable subscription model.

twosugars• 5 years ago

Without AdBlock or browser like brave, Google search is a joke..

ggm• 5 years ago

Wouldn't there be a huge risk to alphabet that this kills the goose that lays the golden eggs?

eyelidlessness• 5 years ago

I almost always get better results from Bing, and while they’re not good they’re not obviously getting worse (barring ahem porn queries which Bing has consistently been better at and still is, you just frequently need to add “porn” to the query now).

hosteur• 5 years ago

I would love a search engine that penalizes monetized content. Content with affiliate links or ads etc.

inasio• 5 years ago

Sure, google is bad, but it's 2022 and searching for shows in Amazon Prime is ridiculously bad. Just yesterday I searched for "Adams family" knowing that they had a movie called "Adams family 2", and it just didn't show in the search results at all, lots of pretty unrelated movies. I had to specifically "search" for the exact movie title.

starky• 5 years ago

Are you sure you didn't just spell it wrong? "Addams Family" brings that up right away on Prime Video for me. Though it would be nice if the search was able to deal with errors like that.

forgingahead• 5 years ago

I want a good search engine for searching internet "chatter" - basically forums, comment sections, etc. I know we have FB, Reddit, Twitter, and so on, but these are all centralised platforms with centralised censorship when the "wrong" comments are made, or the preferred comments are intended to be surfaced/stickied.

It's frustrating to me that I almost never get forum results from Google any more. DuckDuckGo/Bing has been much better in that regard.

jl6• 5 years ago

It would be helpful if these articles/threads included:

1. The term being searched for

2. Screenshot of the actual results

3. Demonstrably better link that doesn’t appear high in the results

I’m not sure Google is capable of doing anything about the issue without some concrete examples.

emrah• 5 years ago

One thing everyone seems to be overlooking is that "quick answers" (or whatever they are called) and the accompanying "related questions" are very good and Google seems to be optimizing for them for the most part.

I agree, Google is no longer good for technical questions or specific queries (like copy/pasting errors messages) but I imagine they crunched the numbers and found that they don't bring much revenue anyway

apatheticonion2• 5 years ago

Can we also talk about the dumpster fire that is YouTube?

I look up anything news related and the entire first few pages are Foxx/Murdoch and whatever other batsh*t content I don't want to see.

I cannot block channels from my search results and I have hardly any control of the content I see. I can't organise subscriptions by folder, I am inundated by "YouTube shorts".

Worst of all, I have a legacy YouTube account with a failed "gaia migration" that I cannot delete because the migration somehow failed and I can't access the content. I have tried contacting their support staff and they told me they couldn't help. It's been 6 years and they haven't been able to do anything about it.

Amazon, Microsoft, Apple - please make a competition service to YouTube focused on the viewer's experience.

xmodem• 5 years ago

Does attempting to delete generate an error page of some sort? If so, you could try triggering it over and over again, as quickly as you can, for at least 10-15 minutes. This will cause their error rate to spike, and likely get the attention of their engineering team.

Hackbraten• 5 years ago

“We have detected misuse of our services. As a consequence, all your Google accounts have been terminated.”

apatheticonion2• 5 years ago

Sadly I can't re-attempt a migration because my YouTube believes my legacy account has been merged (despite it not being merged).

The gaia migration tool simply says "account not available for migration", and that's that.

Gareth3212• 5 years ago

This makes me optimistic. Why? For the longest time, Google has had a monopoly on internet search. This is unhealthy for the entire internet. Google's declining quality opens a rather large door for competition, and I see really promising alternatives such as DuckDuckGo.

Perhaps this is simply an example of the business life cycle in action. Eventually all businesses crush themselves under their own weight and greed.

ronenlh• 5 years ago

If one had all the resources, what should be done to make the web searchable/indexed in a relevant way?

8fingerlouie• 5 years ago

This has been going on for 2+ years now.

When i started using DuckDuckGo, i somewhat frequently had to do a !g to find what i was looking for, but for the past couple of years DDG has been spot on, and Google returns mostly trash. Not that it matters when all the good answers are on Reddit or StackOverflow.

Sadly, DDG lacks behind in the (local) shopping category, at least if you're in the EU.

nathias• 5 years ago

are people really still using google in 2022?

nottorp2• 5 years ago

Hmm I don't think SEO spam and fake reviews are Google's only problem. These have been useless for a long time, but I believe they're altering search results on purpose just to give you more crap to click on.

Random example: paste a stack trace in google, add, say, raspberry pi. First 5 pages of results will be beginner tutorials on how to set up something on your Pi. If you're lucky you'll get something related to your stack trace in the later results. Sometimes you don't get squat.

h0nd• 5 years ago

I tried to google "Mass formation psychosis" yesterday. Guess what happened.

Razengan• 5 years ago

Google Search’s decline has been noticeable for 5 years, particularly in some countries more than others.

csbartus• 5 years ago

Back to the old days. Instead of search, curation.

I have the same problem for years. I simply can’t find the best library / package out there. Neither on Google nor on Github, npm, Duck and co. Therefore I’ve started to build my own personal search engine powered by bookmarks and tags. The irony is that I use Gmail as a bookmarking and tagging provider where search works just perfect.

In addition I subscribe to lots of newsletters, so an inbox search gives the industry pulse + my personal faves.

Outside my expertise / domain I rely on friends. Want a new technical jacket for winter? I ask my friends who do winter sports for ages. In reverse, I give them advice for the web.

And I don’t mind. This is how we went back to ‘personal’ computing.

emptyparadise• 5 years ago

Searching for any PC technical issue solutions is impossible, all you get is grifters selling you PC cleaner junk or some poor overworked Microsoft support tech telling you to run "sfc /scannow" to fix a hardware issue.

988747• 5 years ago

And yet, every time I try to use DuckDuckGo I get frustrated very quickly with a quality of results and go back to Google

pcdoodle2• 5 years ago

It really has been broken. I used to be "the guy" that would find anything.

pcdoodle2• 5 years ago

I would pay $10.00/mo. to make google disappear forever.

ricardo81• 5 years ago

Accuracy aside, I find one of the biggest barriers to easy searching is content padding.

So many queries can be answered in a sentence or two but are spun out into an entire article so that it is more search engine 'friendly'.

Same goes for YouTube videos, pad it out so there's enough time for ads.

I think Google would like something like Pagerank but for authors. Unfortunately Google+ never reached critical mass. Structured data is more prevalent, though, but the accuracy/trustworthiness of it is something that'd have to be taken into consideration.

miav• 5 years ago

The linked tweets imply that this decline in search results was Google's choice, led by desire for further monetization or exec incompetence, but I think Google is simply facing an impossible task.

Receiving an arbitrary question and finding the most helpful site for that question out of the entire web is already nearly impossible.

Now consider the above problem, except the sites you have to search are highly adversarial. More precisely, the internet is roughly divided into people who post useful content and have little interest in SEO and those who only care about SEO and clicks and not about creating useful content. The latter are more motivated and have more resources. For every useful site, they can take that site and create 100 of their own copies with the same content, more aggressive SEO and their own ads.

How is Google, or anyone else, supposed to navigate this landscape?

laboratorymice• 5 years ago

I vaguely recall reading that Google uses time spent on a page/domain as an indicator of the relevance or quality of the search result/content, but I just googled to try to find that source and only found articles stating the opposite [^1],[^2]. Does anyone know if it is indeed used, and if so, why does it not work to demote low-quality SEO-abusing sites? [^1]: https://www.nichepursuits.com/how-important-is-time-on-site-... [^2]: https://ahrefs.com/blog/dwell-time/

janandonly• 5 years ago

The Customer Is King. Google will do anything they can to make their customers happy.

An often made mistake is thinking of yourself as a Google customer.... you are their product (to marketeers) not their customer.

nunodonato• 5 years ago

I'm building a new kind of search engine. Would love if some of you could provide me with some difficult search queries that target very specific content.

apricot13• 5 years ago

I'd noticed this too but I've found that the best results usually are now on page 2 or 3 and the first few pages I've mentally written down as 'sponsored links' so I've just trained myself to ignore them!

mettamage• 5 years ago

Question: what’s a better more qualitative alternative (search result-wise)?

cainxinth• 5 years ago

Has anyone else noticed that Google’s image search, but the standard one and the visual search, have decreased in quality in the last few months? I’ve switched to Bing for images.

api• 5 years ago

Spam won the web just like it won everywhere else. Any open system will be destroyed by spam.

rawoke0836002• 5 years ago

I think many queries to google includes the term "reddit" i know me and my mates all do this a lot more than 5 years ago. The value of 'reddit results' are of course that is mostly real humans (yes not always) providing data and content for a variety of topics.

TheRealDunkirk5• 5 years ago

That's another, adjacent problem. Reddit has obviously given Google a truckload of cash. Almost everything I search for, when I have a question about programming or a video game, includes Reddit results at the top of the list. The problem is that I have NEVER seen a good answer to a question about ANYTHING at Reddit. NEVER. I often use "-reddit" in my searches because of this problem.

rawoke0836002• 5 years ago

Reddit has obviously given Google a truckload of cash.

Source ?

The problem is that I have NEVER seen a good answer to a question about ANYTHING at Reddit. NEVER. I often use "-reddit" in my searches because of this problem.

Interesting ! I guess YMMV :)

roody15• 5 years ago

Have to confirm. I find myself using yandex, (or sometimes https://www.gnod.com/search) to get better results. This was not the case 5 years ago... just my two cents but Google's search has declined from my point of view.

tailspin2019• 5 years ago

The question in my mind, is whether Google has now gone too far in one particular direction for it to be able to ever recover.

It feels like we need a new kind of search engine with a radically different commercial/operating model to take over, with healthier underlying incentives driving the decision making. I don't see the likes of DDG or Bing as being good enough alternatives - they have many of the same problems and aren't significantly different enough from the "Google model".

Perhaps we need a not-for-profit structure for something as important as search. I know Mozilla and Wikimedia have their own problems, but perhaps something along the lines of how these organisations are structured.

Or, we remove the "indirection" baked into the commercial model and allow people to just pay directly for good quality search. Essentially have a model where the "product" is high quality, unbiased search results, as opposed to the product being the eyeballs of the customers using it. The revenue comes from people who can afford to pay for some premium features, where the search results fundamentally are the same for all, but the subscribers (or donors) get added benefits like bookmark syncing, cloud storage or other value-adds.

But, the subscribers effectively subsidise the otherwise completely free search engine for everyone else. The core search results are the same for everyone, paid or not, it has to be a fair, open and non-discriminatory in that respect.

I don't know whether this works commercially, without being pulled into the cesspit of "paid listings" and advertising, or whether it would have to be a not-for-profit/charity setup which also relies on external donors, but that has its own challenges.

As someone who relies on search as part of my day-to-day job, I would happily pay a monthly subscription for a good, clean search engine. Partly out of a moral conviction that an unbiased search engine with good underlying incentives is an important thing for the world, and partly from a more selfish perspective of wanting to improve the quality of a tool that I use and rely on.

I pay Google nothing currently. I considered signing up to YouTube Premium - which I'd be happy to do, if it didn't rely on a whole extra level of Google tracking then being turned on for me (ie. needing to be constantly signed in to YouTube - which then conveniently signs you in to all other Google properties). It feels like even when you pay for a Google service, you're still "the product" and not "the customer".

I occasionally see new search engine projects listed on HN. I wonder if anyone is working on something that may one day be able to knock Google off the top spot - a position which I believe it no longer deserves.

mrkramer• 5 years ago

I'm surprised you are the only person who mentioned this but yea YouTube Premium exists which is ad-free YouTube subscription service.

https://nitter.net/mwseibel/status/1477709354014887938#m:

"What would a paid version of Google Search results look like - where Google can just try to give me the best possible results and not be worried about generating revenue?"

Micheal Google sort of already does this, like tailspin2019 and I said YouTube Premium exists[1][2].

[1] https://www.youtube.com/premium

[2] https://en.wikipedia.org/wiki/YouTube_Premium

liveoneggs• 5 years ago

The last core update needs to be rolled back. I am getting top 10 results / rich-results sites showing adult ads on a regular basis now.

unixhero• 5 years ago

One example: Try to find product reviews of VITRA Office chairs such as EA119 and so on. Impossible!

rdlecler1• 5 years ago

Since 2013 we’ve been publishing AgFunderNews as the site of record for investment and entrepreneurship in foodtech and agtech. For whatever reason we were never able to get into Google News but other were who write summary articles on the articles we publish. Similarly, Google will defer to a site like Forbes which has no real authority in our space. It’s really really bad. There is good info out there but increasingly Google is not able to surface it.

mrlanderson692• 5 years ago

If anyone has the skills / wherewithal to work on this problem please send me an email asap (my email address is in my profile.)

And as I've stated, I will give you $500 if you do not like the demo of what we are building just to show you that I am not f*&%ing around with this. Our small team is growing very carefully in a well-defined niche for this.

ckmar• 5 years ago

What are your thoughts on using social trust for this? EG https://app.kujo.com

mrlanderson692• 5 years ago

We are working on exactly this problem. IF anyone wants to see a demo please email me.

husamia• 5 years ago

The econiomic model has to change. The next Google will be one of those decnetralized web3 app.

neets• 5 years ago

I am going to go out on a whim and say that I think this is intentional. There are a lot of power games being played these days. I am reminded of this quote from Chomsky,

"There are huge efforts that do go into making people, to borrow Adam Smith’s phrase, “as stupid and ignorant as it is possible for a human being to be.” A lot of the educational system is designed for that, if you think about it, it’s designed for obedience and passivity. From childhood, a lot of it is designed to prevent people from being independent and creative. If you’re independent-minded in school, you’re probably going to get into trouble very early on. That’s not the trait that’s being preferred or cultivated." [1]

Google is how the masses are "educated" these days. There is also the narrative of how students, high school, university, and collage are just googling everything for their tests and exams and not learning everything. (I am one of them)

I am sure SEO spam is part of the problem but there are always multiple narratives going on.

[1] https://chomsky.info/warfare02/

amai• 5 years ago

We need a classic spam filter for search results. I can mark emails as spam. Why can I not mark search results as spam? This could be used to train a naive bayesian filter similar to the spam filter in my inbox.

Google no longer producing high quality search results in significant categories

Related Stories