Rapgenius Uniques Fall 90%

https://www.quantcast.com/rapgenius.com?

by conover • 13 years ago

62 118 13 years ago

Ok, how can Rap Genius recover from this? They're one of the best (and least shady) lyrics sites on the net. They acted immorally, yes, but is justifiable to summarily execute their company?

Google has a domineering position in the market, and it's a bit ironic that people were quick to call for Microsoft to be split into multiple companies about a decade ago, yet nobody bats an eye when Google exercises their power. (I personally don't believe either company should be split; just pointing out an interesting parallel.)

So what can Rapgenius do? Are they screwed?

EDIT: Also, how long does the penalty last for? I think people are assuming 30 days, but is that really the case?

Even still, 30 days is a long time. They may not be okay afterwards. Whereas, say, a week-long ban would've had the same deterrent effect.

obituary_latte• 13 years ago

It's less execution and more suicide isn't it?

rickhanlonii• 13 years ago

Exactly. I know the scene here feels bad for RapGenius because they're putting themselves in their shoes and appreciating how devastating a penalty this is. But I don't feel bad for them whatsoever, and neither should anyone else.

This is a penalty that they deserve. They took a risk and they lost. They knew what they were doing and the potential destruction of their traffic, and they did it anyway.

You don't get less time in the penalty box just because you're the favorite or best player on the team. In fact, you have a greater responsibility to play by the rules and I would make the argument that your penalty should be even greater.

Besides, if the refs played favorites, the refs would have no credibility.

JohnTHaller11• 13 years ago

There is no execution going on. They knowingly engaged in blackhat SEO. Now they are paying the rather well-known penalty. They need to wait it out while that penalty is active (likely slimming down their burn rate) and focus on gaining customers through other avenues.

Or kill their brand and reboot as a different one with a different URL on a different IP.

TrainedMonkey• 13 years ago

Do not underestimate domain age/link age in calculation of pr. Moving to a brand new domain would forfeit all of that.

joeblau• 13 years ago

They just need to follow the rules. I think once Google takes them out of the sin bin, they should regain their ranking.

ddoolin• 13 years ago

I agree with this. They should be fine once they're reinstated.

scott_s• 13 years ago

I think that tech people talk about Google's power constantly.

adventured• 13 years ago

Before Google it was Microsoft. Before Microsoft it was IBM.

adamnemecek2• 13 years ago

Maybe name and domain change?

ceejayoz8• 13 years ago

That puts them at PR0, too.

dsil• 13 years ago

I expect them to go back to right where they were once their penalty is lifted. Assuming most of their visits were google searchers clicking on the top link, nearly all of those visits/day will come back once they're the top link again. Remember that a huge majority of people have not heard about this controversy. The hacker news sort of community is not their primary audience.

It's possible they'll be bigger, what with all the free (though bad) press they've gotten.

JohnTHaller11• 13 years ago

They may not be back at that top spot, though. It depends on how much of their ranking was organic and how much of it was due to the cheating.

mtrimpe4• 13 years ago

A friend of mine ran a long tail content generation service, much like HitTail, and recently had all his clients penalized regardless of the quality of the content they wrote for those long tail keywords.

Apparently they already anticipated something like this because several other companies were being manually penalized over the past year and half following an algorithm that seemed like: "Which big AdWords spenders cut down significantly on their budget? Are they getting it organically now? Let's cut that off."

Several of those companies are suing Google for violating EU regulations, so hopefully something will come out of that.

The first step will probably be that penalizations can be fought through the EU judicial system, but it will be a while before we get jurisprudence on that.

I don't know how this would work in the US legal system but, regardless of what the outcome would be in Rap Genius' case, it's probably time for similar jurisprudence to be established in the US.

cycrutchfield6• 13 years ago

Adwords spend is completely unrelated to search ranking.

oliwary• 13 years ago

I think the fact that they did lose as much of their uniques means that there is not a lot of brand-loyalty in the market. Then no people would form loyalty for competitors that are now over RG in the google results and they would be back once the penalty is over or RG works things out with Google.

cdgreen1• 13 years ago

It's just a matter of how do you quantify a crime against the almighty Google? I understand that Google have rules and regulations in place to protect themselves and consumers from these "online black hat thugs" but even the justice system have standard jail time for certain crimes. Now maybe there is a "standard jail sentence" in the Google justice system for voliating the rules, but what if there is not one? Do we allow Google to impose a death sentence for whatever they see fit?

minimaxir2• 13 years ago

What's interesting is that the number of uniques is on a downward linear trend after the initial penalty.

sscalia2• 13 years ago

"Don't be evil"

Rapgenius got punished for playing the game better than anyone else.

Who's to say this couldn't happen to highly relevant sources of information? Wikipedia? Underground sites?

This is the equivalent of censorship.

minimaxir2• 13 years ago

What?

The "game" RapGenius played was the act of being evil.

petercooper• 13 years ago

the act of being evil.

Let's get some perspective here. They were offering to promote people's pages in return for them linking back to them. Far from being evil, this isn't even illegal.

JohnTHaller11• 13 years ago

First, you don't understand what censorship is.

Second, RapGenius cheated the system. They knew they were cheating the system. They violated the public guidelines that tell you not to cheat the system. And now they are paying the price.

Why is that so difficult to grasp?

goldenkey3• 13 years ago

In addition, they had no furtiveness; it appeared they thought this was something that required no concealment. I'm guessing they don't have a media relations employee, because any decent quality control would have said direct emails from @rapgenius to random folk about a seedy scheme aren't the best idea. At least a proxy should have been used..

Their emails were wanton risk, the wording had no stealth; they got what they deserved.

trevelyan• 13 years ago

Highly relevant sources of information don't show up in the search results anyway.

sejje4• 13 years ago

They most certainly do in the vast majority of my searches.

quack• 13 years ago

Wikipedia isnt gaming the system to make wikipedia relevant. Theyre not ramping up efforts to get inbound links from random blog websites. If anything this shows that RapGenius's numbers were hyper-inflated by SEO games rather than user relevance.

cycrutchfield6• 13 years ago

"This is the equivalent of censorship."

How is penalizing them for engaging in shady SEO tactics considered censorship?

sejje4• 13 years ago

When I google "eminem lyrics" they don't show up in the results. Fine, whatever, seo penalty.

When I google "rapgenius eminem" they don't appear in the results. That's pretty close to censorship.

I'm not necessarily against it, but I don't disagree with the usage of the term.

sejje4• 13 years ago

I think they got punished for playing the game worse than anyone else.

Blackhat SEO out in the open is not a good game.

ceejayoz8• 13 years ago

Rapgenius got punished for playing the game better than anyone else.

I'm sorry, what? Didn't they lose the game?

colechristensen• 13 years ago

They weren't 'playing the game better,' they were cheating. It wouldn't ever happen to Wikipedia and you know that the comparison is absurd.

It isn't the equivalent of censorship, it's rewarding the good behavior of the competition.

goldenkey3• 13 years ago

It wasn't so genius to be expository to random people via email, about a link-back scam. Sorry RapDingus, it's over; maybe next go around you'll have less hubris and daringness in using unethical practices.

http://jmarbach.com/rapgenius-growth-hack-exposed

sscalia2• 13 years ago

If you think this isn't happening all the time, everywhere -- you're mistaken.

goldenkey3• 13 years ago

Not many funded startups are dumb enough to publicly email folk asking for links in return for tweets/linkbacks. That's pure stupidity; might as well upload a video on a bad political stance with your brand overlayed, because the worms don't stay in the can with any kind of public communication. They should have been more clock-and-dagger with their scheming -- their hubris was their downfall.

Of course many spammy sites use this practice but they don't have the respect from the tech community that Rap Genius warrants as a Y-Combinator funded entity. They lost that, and it's going to be a steep climb for them to find talented folk that want to be part of a seedy operation.

boxy2• 13 years ago

People rob and steal, does that make it acceptable for you to do it? People are racist, does that make it acceptable for you to do it? People don't give up seats for pregnant and old women, is it acceptable if you don't? Not everything is black and white, but this is. What I'm getting at sscalia is that the small minded way people like you think is something that's wrong with this society. Fix up.

brianpgordon• 13 years ago

It's pretty scary how much Google controls the internet. I wish Bing or Yahoo or DuckDuckGo weren't so much worse than Google, or I'd happily use them.

captainmuon4• 13 years ago

Give me just 20-30 million dollars, legal protection from any patent claims, and (providing my search engine works well technically) make it the default in a small percentage of all Firefox installs.

What I'm trying to say is, Google doesn't have the monopoly because they're the only ones who can build a good search engine. The have the monopoly because they do have good technology, but also because of path dependence, and because there is no political effort to build up a competitor.

I already posted a couple of days ago, there are a bunch of ways governments could support competitors when one search engine controls a large fraction of the market, say more than 2/3. For example a special monopoly tax, subsidies for competitors, etc.

One thing I think would be nice is to make search engines offer a free/very cheap API (as long as they have > XX % market share). Anybody can come along and build a Google + Bing + Pokemon-Wiki meta search engine. If you tune it to be profitable, you'll have a lot of competing search engine frontends spring up. But you'll also have a lot of search engine backends (and not just Bing, Google, and Yandex), because its suddenly profitable to build verticals (e.g. indexing just tech news or recipies), which require less resources technically - because the vertical backends get traffic through the competing frontends.

And I could choose a frontend that gives me Google-like searches, or one that interprets terms more literal (like when you use +"term" in Google), which would be great for programming related stuff. There could be another one focussing on social media, or local stuff, etc..

vijayboyapati• 13 years ago

Not only is this bad for RapGenius, it's bad for Google, and most importantly bad for Google's users. Yes, RapGenius screwed up, and they've paid the price. But they've publicly recognized what they did and apologized. Google really should reinstate their ranking, because people searching for lyrics right now are being given the huge disservice of using the ultra-spammy lyrics sites from days of yore.

JohnTHaller11• 13 years ago

They haven't paid the price yet. RapGenius knowingly engaged in blackhat SEO and broke the public rules. The punishment is several months in SERP Siberia. RapGenius hasn't paid that yet, but they will.

ceejayoz8• 13 years ago

Google really should reinstate their ranking, because people searching for lyrics right now are being given the huge disservice of using the ultra-spammy lyrics sites from days of yore.

A short penalty of a few days or weeks isn't much of a deterrent. It may suck for people searching lyrics now, but making the page rank penalty effectively toothless would suck for everyone.

bonemachine• 13 years ago

The hell-banning will probably only be temporary (30 to 60 days).

And RG didn't just "screw up" by trying to game the system; they were blatantly cocky about it, so that's probably why Google felt they needed to step in and teach them a thing or two. Let's hope the lesson sticks, at least.

owenmarshall2• 13 years ago

it's bad for Google, and most importantly bad for Google's users

BS.

What's good for Google is showing people that if you engage in bad practices your shit will get nuked from orbit, even if your site is better than the competitors.

You'd better believe that companies are going to avoid blackhat SEO in the future after this, and that's absolutely in Google's interests.

hobs• 13 years ago

Yeah, it would only be actually "bad" for google if people were switching search engines en masse because they couldn't find the lyrics they wanted.

I think that anyone who characterizes google as needing one site over needing people to not blackhat will find that google really doesn't need said site. (Maybe Wikipedia is an exception to this rule?)

zorpner• 13 years ago

More specifically, companies are going to avoid entering markets (e.g. online lyrics databases) where Google has demonstrated an unwillingness to appropriately punish established players who use the same black hat practices, since it's impossible to compete in the organic rankings against them.

Harj• 13 years ago

What's good for Google is doing what made them successful, providing the best search results. We now know one type of search they're no longer doing that for.

codygman6• 13 years ago

They aren't above any other good site "just trying to get traffic" and not be "trampled by blackhat seos" and should face the same penalty.

I applaud Google for this, and that's the first time I've applauded Google in a while!

rok3• 13 years ago

Google should wait for RapGenius to a) clean up their backlink profile b) Submit a request for review just like everyone else.

I work at an online marketing firm and I've seen dozens of companies come to us for help with the process. RapGenius shouldn't get preferential treatment.

jonknee2• 13 years ago

people searching for lyrics right now are being given the huge disservice of using the ultra-spammy lyrics sites from days of yore

AZLyrics is a lot more appealing to me. Much lighter weight.

jmduke• 13 years ago

Hypothetical question: if we ("we" being nebulous) were to unveil that every other major lyrics site (azlyrics, metrolyrics, stlyrics, lyricsfreak, lyrics365, etc. etc.) uses the same SEO tactics, would they receive the same attention and punishment?

cycrutchfield6• 13 years ago

Yes, but those sites have more sense than to trumpet their affiliate link scams on Twitter

zwily• 13 years ago

I think everyone pretty much already assumes that all the other lyrics sites are as shady on their SEO as they are prolific in their on-page advertising. RapGenius was kinda held to a higher bar.

captainmuon4• 13 years ago

Someone should just do blackhat SEO for a bunch of those sites, and then Google would have to punish them too (because who could proove that they didn't do it themselves?)

Or taking it one step further, use these tactics to promote a controversial institution or politician (without them knowing). Then "leak" it to Google, so they get punished. Then claim Google is censoring them. Finally enjoy the outrage :-)

poopsintub5• 13 years ago

Domain and page authority weigh in on how much you can get away with link tactics imo. Sites that have been around for a few more years would likely withstand any blackhat tactics a third-party would shoot at them.

adrr• 13 years ago

Google webmaster tools lets you disregard back links which removes any benefit or penalty of the links. So if someone added your site to a known link farm, you can disregard those back links from affecting your ranking.

loupeabody• 13 years ago

I tend to feel like they wouldn't. Maybe RapGenuis's more refined image and branding yields a harsher judgment in this case as they probably perform well in many other aspects of SEO.

The link selling, not the meta data of their pages, was what caused this. It's likely that these other lyric sites do these sorts of arrangements in a less public way, whereas RapGenuis was using Twitter, of all places, to sell links.

JohnTHaller11• 13 years ago

The punishment is about the same for all websites that engage in blackhat SEO. At least, once they clean up their acts. Until they clean up their acts, the punishment remains in effect. This can be difficult when you pay for nebulous backlinks from spam sites in a network, for instance, as played out with the Anglo Links takedown Google did last month: http://searchenginewatch.com/article/2319130/Did-Anglo-Rank-...

Mind you, this is not the exact behavior RapGenius engaged in. They paid for backlinks, but they paid legitimate sites directly for links in blog posts. Either way, though, it's still blackhat.

It's a really interesting world if you feel like reading up on it at some point. I've always done basic SEO (read: real, legitimate, basic stuff that I'd never even advertise as 'whitehat SEO' since that nearly always means 'blackhat' but we'll pretend it isn't) for my personal sites and client sites over the years. But you're going up against these 'players' who routinely setup networks of spam link sites, backlink payment networks, etc. Granted, most of those are pump and dump. They build them up, make their money, get penalized, then burn it to the ground and move on to another domain. Still, it can be disheartening. And it may tempt some people to take the blackhat approach. But, one look at the penalties should be enough to change their minds.

zinssmeister• 13 years ago

It's quite painful to watch these guys take this beating. What they've done wasn't nice, but the consequences are extreme. What is most concerning is the monopoly Google Search has over driving traffic/revenue for most sites.

istorical3• 13 years ago

SEO is such a fickle beast.

To everyone here who's saying they brought this upon themselves, when rules are unenforced and all your competition breaks them, you've already hamstrung yourself if you don't break them too.

If everyone in the MLB is juicing and a rookie gets caught, you don't say "what an idiot", you look at either 1) enforcing the rules on everyone and broadly changing the culture or 2) question whether the rules make sense in the first place.

codygman6• 13 years ago

No. They broke the terms of service. They got banned. End of story.

istorical3• 13 years ago

What's it like seeing the world in black and white?

pnathan• 13 years ago

meta-note: this is a great real-world example of the Lawful vs. Chaotic arguments.

JohnTHaller11• 13 years ago

It's not that fickle. Even if all their competitors are engaged in blackhat SEO and buying links from other blog sites (using cash, tweets, etc). RapGenius was the one who publicly boasted about it. Which caught the attention of Google. Which earned them the smackdown. That's not fickleness at work. That's... what's the opposite of genius?

almosnow• 13 years ago

what if... rapgenius is a beautiful pr stunt cooked by google to show everyone how valuable it is for websites/startups?

Greenisus2• 13 years ago

What should Google do in this situation? I fully understand them murdering Rap Genius in their indexes, but it also hurts Google since Rap Genius actually has the best content.

First result for "jay-z holy grail lyrics" Google: http://www.azlyrics.com/lyrics/jayz/holygrail.html Bing: http://rapgenius.com/Jay-z-holy-grail-lyrics

The Bing result is better (though both are valid).

I would suspect that most sites pulling these sort of SEO scams are going to have lousy content, so the rule makes sense, but perhaps there should be some sort of human element to it as well? That's not scalable, though, so I'm not really sure what the best answer for Google is on this one.

ceejayoz8• 13 years ago

What should Google do in this situation? I fully understand them murdering Rap Genius in their indexes, but it also hurts Google since Rap Genius actually has the best content.

It helps them in the long term, as it dissuades sites from using blackhat SEO if they know their company can suffer severely for it

dkuntz25• 13 years ago

Is the Bing result really "better"? Lots of people would rather not have the annotations...

Additionally, with AdBlock on, AZLyrics makes only 21 requests and takes 1.03 seconds to load completely on my machine. RapGenius (still with AdBlock on) takes 27 seconds to load, with 208 requests. Based solely on that I'd rather use AZLyrics than RapGenius.

Greenisus2• 13 years ago

Yeah, I suppose it's subjective. I personally think the RapGenius result is better, but you make a valid point about the performance and simplicity of the AZLyrics result.

jonknee2• 13 years ago

Luckily their revenue has remained unchanged... $0. This could have been a financial disaster if Rap Genius was a real business, but you need to have revenue for that.

jonathanjaeger2• 13 years ago

Many people would kill to have a site with millions and millions of uniques. If monetization HAD to be the primary focus, they could make money. Growth and product is the top priority. Unfortunately for them one of those is suffering right now.

JohnTHaller11• 13 years ago

A site with millions of uniques doesn't = revenue. Nor does it equal success, interest from investors, etc. Source: running a ~2m unique site right now.

maaaats• 13 years ago

I've missed some news lately with Christmas and all. Any context on this?

conover• 13 years ago

Context: https://news.ycombinator.com/item?id=6963365

champion• 13 years ago

damn, they got CUTT

boxy2• 13 years ago

The title of this post has annoyed me.

physcab• 13 years ago

They'll recover. As much as I dislike their PR attitude (whether an act or not), they have a good product. I still listen to rap music, and I still want to know the meaning of the lyrics, so I'll go directly to the site if I have to. Google made it one click easier, but whatever, I have time.

Companies have survived much worse blows. I worked for a music company that was not only being sued by every major record company, they were also taken out of all app stores. They persisted, and today I still listen to music from their site, even almost a decade later.

spindritf2• 13 years ago

To recap. Google's algorithm is terrible, and despite having more PhDs on board than my uni, can be played by anyone with two brain cells and a mildly popular Twitter account.

To rectify this, they crippled my search results by penalizing Rapgenius and instead pushing some AZlyrics junk that not only lacks annotations but is inaccurate.

Seems like an easy fix, for the end user at least, add a keyword search for 'lyrics' to go directly to Rapgenius, right? No. Here's a search for smoke on the water

http://rapgenius.com/search?q=smoke%20on%20the%20water

First result? Pro Era – Like Water. Second? Childish Gambino – I. The Worst Guys. Third? Smoke DZA – Diamond.

Arguably the most popular song in the history of rock and roll? Sixth.

Those are some of the smartest people on the planet. I'm not being sarcastic. How depressing.

dkuntz25• 13 years ago

They didn't really push AZLyrics, or any other lyrics site up. Those rankings haven't changed. Their position relative in the list you see based on your search terms has, but that's relative.

Why should Google give special dispensation to RapGenius? Why should putting in something like "lyrics" or related content to someone being penalized get them to show up higher in the results?

Why should Google be RapGenius' search engine? Why shouldn't RapGenius make their search better?

spindritf2• 13 years ago

RapGenius should have a better search. And Google shouldn't give special dispensation to RapGenius, it should return best possible results.

Which they don't after excluding RapGenius. They purposefully broke their search results to protect their broken ranking method.

cycrutchfield6• 13 years ago

What are you going on about? I don't think you have the slightest idea about how Google's ranking algorithm, or for that matter any ranking algorithm, actually works.

captainmuon4• 13 years ago

Google is resting on their laurels. I'm convinced their actual algorithm hasn't changed much in the last few years, despite announcements claiming otherwise.

They changed the handling of search queries (searches are interpreted much less literally than a few years ago). They changed the way that pages are indexed, and changed the intervals to be smarter (faster for news sites, slower for sites that don't change). They now even "render" the pages partially to account for CSS and JS. They have smart stemming and synonym algorithms when building the indexes. But the actual algorithm - "PageRank" and the parts surrounding it - hasn't changed much I think, except for some parameter tuning.

Instead, they probably have a pile of semi-automatic "filters" to deal with spam and blackhat SEO. Engineers write a filter in code, test it against a test index, and then it goes into the new index version. I believe they are going more and more into manual intervention, and that they can now directly penalize/push sites without going down to the indexing level.

I can't prove it right now, but I'm pretty sure that the first SERP for popular search terms is actually manually curated, with the really organic links coming later. (I'm still trying to figure out a test methodology to analyse it :-).)

siegecraft• 13 years ago

First off, who cares if the top result now gives you no annotations? No one cared about annotations until rapgenius started adding them so people could know what the more obscure portions of rap lyrics meant and then they gamified the shit out of it so that every single word in a song would be annotated so that you now know the meaning of "I set fire to homes, arsonist" is because it helpfully explains that an arsonist is someone who sets fires. And then put annotations everywhere even on their stupid blog posts, it's like they just discovered HyperCard.

And how to find smoke on the water? Maybe put some quotes around it? http://rapgenius.com/search?q=%22smoke+on+the+water%22

clarky07• 13 years ago

It seems somewhat unreasonable to manually penalize a website like this. If you can't figure it out algorithmically then it can't be that bad. Major companies (much bigger than Rap Genius) buy links all the time. How do I know? They have asked me for them. I used to run a financial blog and I know a certain company that bought links from pretty much every personal finance blog that would sell one to them, and I know this for a fact because I was in a network of blogs and talked about this on forums as a way to make money. I also have the emails to prove it.

Why doesn't this company get penalized? I have no idea. I guess nobody decided to write a blog post about it, it didn't get picked up on Hacker News, and Matt Cutts hasn't noticed it yet. Seems like if this is possible there should be a reverse black hat market springing up soon. Do some really bad black hat SEO for your competitors and then get them killed by Google.

Rapgenius Uniques Fall 90%

Related Stories