Mahalo's disappearing noindex tags

http://smackdown.blogsblogsblogs.com/2010/03/08/jason-calacanis-makes-matt-cutts-a-liar/

by dazz • 16 years ago

193 70 16 years ago

smackdown.blogsblogsblogs.com

chaosmachine• 16 years ago

I'm sure it's just a bug, and they're working to fix it real soon now... ;)

die_sekte• 16 years ago

I am just waiting for Calacanis to show up here and say just that.

I have never seen anybody else lie that openly and get away with it.

froo4• 16 years ago

>I am just waiting for Calacanis to show up here and say just that.

Lets not forget he'll use his obligatory "Peace n love" signature...

omarchowdhury• 16 years ago

When will we stop hearing about Calacanis and Mahalo? The only thing I take away from the whole thing is how effective manipulating search engines can be. And I already knew this. From blackhat SEOs.

The only thing the blackhats haven't taught me is how to get into the Google index and stay there - how are you doing it, Calacanis?

vaksel4• 16 years ago

when Google decides to apply it's rules.

Mahalo is a black hat SEO site, that's given preferential treatment by Google. Any other black hat SEO site would have gotten delisted a month after launching.

sh1mmer• 16 years ago

When people stop up voting it...

jasonmcalacanis2• 16 years ago

This is getting a little silly I agree. I guess the SEOs are not going to stop so we're all going to have to suffer as they create spam pages in Mahalo and then point to them and say "Calacanis allows spam!"

Update:

1. Did we have show short pages in Mahalo? Yes.

2. How did they get there? Our users start topic pages and don't finish them.

3. Were they getting traffic? Not really... like < 6% of our traffic was from short pages. We know this because we've not deleted them or greatly improved them.

4. Why does this keep coming up with the SEOs? Because I called SEO bullsh#$t back in 2005 when someone asked me at a conference why Engadget and Joystiq were doing so well with SEO.

5. Do you still think SEO is bullshit? No, I don't. Back then I didn't know what SEO was and assumed it was spamming google so I said, "oh, that's bs." This was true at the time. At Weblogs Inc. we did zero SEO and we had amazing search traffic. Why? Well, it was later explained to me that we made a lot of content (20 posts a day across 50 blogs = a lot of posts), we got a lot of comments on them and we used keywords in the titles of the posts before anyone else used them. Today's I don't think SEO is BS. I think SEO is an essential part of building any startup just like building a viral loop, doing PR or hiring great people.

6. Are you sorry to the SEO community for hurting their feelings. Yes, I'm really really sorry.

7. I found a page that has no content on it? yes, that will happen from time to time... when we find them we delete, redirect or expand them. These will be in the < 1% of our corpus range. If you do find one email me jason at mahalo.com.

8. don't you have tools to do that? yes and no. we're building them. we need to balance turning off pages people are planning on building out vs. ones that are abandoned. So, you will see nofollow on pages with no content, and no follow on pages with < 100 words if they are older than X day, etc. This is all being built now.

That's it... there is nothing bad going on here. We're good people doing good work. Hundreds of people are making real money to take care of their families at Mahalo and I wish you guys would stop attacking them.

If you're pissed at me take it out on me, leave Mahalo out of it.

.... or don't. Who cares. This is so biased and absurd right now I don't think any is really taking it seriously any more. Aaron had a handful of good points last month, but now folks are just doing this to linkbait for their SEO sites.

great... trust me, this is making my experience reading HN annoying too. If HN wants to filter out "calacanis" and "mahalo" from HN i'm all for it. Clearly the system here is being abused.

mvandemar3• 16 years ago

Jason, you are actually full of crap. For starters, you are mistaken, or pretending not to know, as to why this keeps happening to you. You brought this on yourself, and on Mahalo.com, by unsubtle spamming and boasting about it, and then being rude to boot. You can whine all you want about the perceived injustice of people calling you on your crap, but the fact remains it is your actions that are getting called out, nothing more and nothing less.

This isn't about a small handful of pages that don't have content on them, and if you read the posts you were rebutting instead of trying so damn hard to make yourself sound better then you might actually have information you could use to improve your site.

The vast majority of the pages on Mahalo are empty pages devoid of human generated content. It is not like I am straining my ass off to find them, either. Click on this link, scroll down the page, click through to 20-30 pages (the ones with all lower case titles demonstrate what I am talking about most often, focus on those), and LOOK AT YOUR WEBSITE:

http://www.google.com/search?num=100&hl=en&safe=off&...

In order to appease Matt Cutts, who gave you the courtesy of warning you instead of banning you outright, you did in fact add noindex to all of those pages. 1 day later, after he saw that it was there, you removed it again.

Pretending not to know what is being talked about and making shit up in an attempt to garner pity for being picked on is not helping your cause any.

cscotta• 16 years ago

Would you care to comment on the "noindex" issue? It would be interesting to hear how these are applied and removed (algorithmically, by hand, etc?).

beilabs• 16 years ago

Afraid I only agree with one thing above:

"If HN wants to filter out "calacanis" and "mahalo" from HN i'm all for it."

Hear Hear!

alain940403• 16 years ago

Shame on Google algorithms for not automatically being able to detect a page without content.

Over the years, my view of the Google's ranking algorithms went from dream-like (they are so smart they understand everything) to infant-like (just a bunch of keyword counters).

Obviously there has to be huge complexity involved, so my current view must be pessimistic, but still...

Just one (unrelated) example: search for "leader in online used cars" in Google. Does every human understand what I'm looking for? Here's search result #5: "Indian holy leader and BA stewardess arrested over prostitution".

michael_dorfman• 16 years ago

Damn, you're going to feel pretty foolish if it ends up that either the Indian holy leader or the BA stewardess are among the current leaders in online used car sales.

alain940403• 16 years ago

I would :-)

That being said, just to add fuel to the fire, I did the same search on bing. All results were relevant, unlike Google's. But none of them gave me the answer I was looking for.

scott_s3• 16 years ago

Google is a tool, and like all tools, you need to learn how to use it. Two of your keywords are unnecessary, and I think that caused the problem. First, you don't need "leader." If you're looking for the top of something, you already get that information from the ordering of results. Second, "online" is redundant, since anything you look for will probably be "online." (And I suppose you don't need "in," but I don't think Google used it anyway.)

So then you're just left with "used cars." That result page looks reasonable to me.

alain940403• 16 years ago

You make my point better than I could! You just explained to me that my query, which every human understood perfectly, is stupidly understood as "used cars", so Google is a basic keyword algorithm and this still doesn't explain result #5.

Really, I'm smart enough to understand your explanation of how the tool works. The problem is that I'm in the 99th percentile in the field of "computer keyword understanding" over the general Internet population. So what does that say about the tool?

nfnaaron• 16 years ago

"First, you don't need "leader.""

Unless he wanted to find a page writing about "leader," like a review, or even an ad, rather than the top google-ranked used car page itself.

krakensden• 16 years ago

The problem is that PageRank was a beautiful solution that pretty accurately modeled the web, as long as no one knew about PageRank.

jacquesm• 16 years ago

In electronics it is a well known fact that it is impossible to measure a system without influencing it.

I think is no way you could ever build a successful search engine ranking algorithm that will not influence the web to the point where that algorithm will lose a lot of its initial edge.

MaysonL• 16 years ago

Interestingly, Googling your string, in quotes gives only ads and your post. ;-)

raganwald• 16 years ago

Who is Jason Calacanis and why is this particular but of SEO notable? What is aspect of this should "gratify my intellectual curiosity?"

jcromartie• 16 years ago

Jason Calacanis is a guy who runs a sweatshop building a website you've probably never been to and never will because you're a smart person who doesn't fall for spam.

He also has made a career out of being a professional asshole and making fun of "lifestyle businesses" that make products that people like and pay for. He thinks that if you're not a workaholic and want to enjoy your short time here on Earth then you shouldn't be working with startups.

josefresco• 16 years ago

Keep your personal opinions out of this debate, please.

This is who Jason is for those who do not know why his 'bad behavior' is so watched and talked about: http://en.wikipedia.org/wiki/Jason_Calacanis

dschobel• 16 years ago

I don't particularly care about Calacanis (sorry Jason) but I'd say gaming Google this thoroughly is pretty newsworthy.

Or at least it was the first three times we read about it.

SwellJoe• 16 years ago

What's amazing, to me, is that we're still reading about it several months into the story. Mahalo is a known black hat spam site, with despicable tactics, and Google continues to apologize on Calacanis' behalf and send traffic to Mahalo.

I think that's actually the interesting part of the story: How Calacanis has snowed so many smart and decent people into doing his bidding.

tptacek• 16 years ago

He's allegedly building a business on scraping sites and re-hosting their content to exploit long-tail search. SEO is in general germane to Hacker News. What Google does (or does not do) to deal with Mahalo will be relevant to many YC-style startups.

benatlas• 16 years ago

I though it was Matt's job description to noindex pages? Oh, I get it now, he delegates, let the patients run the asylum...

sp332• 16 years ago

http://en.wikipedia.org/wiki/Noindex

jaxc• 16 years ago

If Mahalo is alleged to be a scraper site then just let Google handle it.

If you think about, if Google delists the site then thats probably their business model gone because Google is like 60% of search Market in the US and higher else where so if Google delists them and take away their Adsense then they are in trouble.

So let them taunt Google all they like and then see what happens when the Google Dragon wakes up because it won't be pretty.

(Edited for typos and readibility)

byrneseyeview2• 16 years ago

Aaron is basically acting as their Ombudsman right now. Google will probably lose some serious revenue from that, because they get a big cut of what Mahalo generates. If Google lost that ad revenue for some other reason, the person responsible might well be fired for it.

When Aaron Wall hounds them over it, they'll be more likely to make the right decision.

jfarmer2• 16 years ago

Google does something like $18Bn per year in revenue and of that about $750MM-$1Bn is from ads on non-Google-run sites.

So, I think they care about Mahalo as a source of revenue about >< that much. They're probably more worried about possible anti-trust actions if they just drop Mahalo -- I bet that would shave more off their market cap than losing Mahalo as a customer.

browser411• 16 years ago

Goog's 60% market share is understated, too. Yahoo and Bing often point search results back to their own networks (Yahoo & MSN respectively). So third parties like Mahalo probably see Google driving much more than 60% of their search traffic on average.

vaksel4• 16 years ago

For me Google usually brings 88-92% of SE traffic, that's the same for most people. Google is understating their market share, because essentially they have a monopoly, and if that was widely known, they'd be regulated in no time.

utnick• 16 years ago

The spam page example given in the article is: http://www.mahalo.com/aaronwall

Question: how does somebody end up at this spam page? Google seems to do a good job of filtering stuff like this. It wasn't in the first 10 or so pages of search results for "aaron wall"

byrneseyeview2• 16 years ago

First: I highly recommend Aaron Wall's rank-checker tool:

http://tools.seobook.com/

Second: the idea is not to rank for every page. But the marginal cost of one more page is basically zero. So if even 1% of them rank on page one, the return on investment can be pretty high.

Calacanis is in a tough position, to be honest. He can dial down the quality and automatically get more revenue for no extra money. But he can't reduce the capital expenditure he made to create this system in the first place.

It's sort of like being a janitor, but having a great connection for huge quantities of black tar heroin. You could be a really stand-up guy, but after a while, all the easy money gets tempting...

mvandemar3• 16 years ago

Sorry I did not answer your question earlier. I used that particular page in the post because it was an example of how these autogenerated pages come into being in the first place, from a post I did back when I first called Jason on the practice. It was the actual page that I had seen the noindex tag on after Jason added it. You can get to any of these autogenerated pages by running a query on Mahalo itself.

It was never an issue of how many of the empty pages were ranking per se (although quite a few do, just not that one), but rather of the sheer numbers of them that were collecting enough tiny quantities of of PageRank each, that was then (and is still, as a matter of fact) adding up to massive amounts of self-sustaining ranking power:

http://smackdown.blogsblogsblogs.com/2010/02/22/apparently-j...

I hope that helps.

ryanb• 16 years ago

I think this whole thing would have blown over if not for Jason's incredible ego and his "I can do whatever I want" response to the whole issue. Pissing off the geeks = bad idea.

houseabsolute• 16 years ago

Posted this over on that blog, but I thought it might be interesting to HN readers too:

I also noticed this snippet of code:

  if(url.indexOf(”google.”)!=-1||url.indexOf(”search.yahoo”)!=-1||url.indexOf(”search.live”)!=-1||url.indexOf(”bing.”)!=-1){
    if($(”#adsense_2719325″).parents(”#right-column”).length > 0 && !isImage){
      $(”#adsense_2719325″).addClass(”afc_wide”);
    }
    $(”#adsense_2719325″).html(ads);
    $(”#ads-section-2719325″).show();
  } else {
    $(”#ads-section-2719325″).hide();
  }

Does that mean that he basically shows different content to people coming from search engines? Is that kosher?

vaksel4• 16 years ago

showing/hiding ads is perfectly fine.

What you can't do is use that to keyword stuff your site.

prawn• 16 years ago

On one of my sites, I used AdSense channels to learn more about the CTR from those arriving from Google vs regular users of the site. The CTR from incoming Google traffic was 10x higher, so I show ads to that traffic but not to the regulars - keeps them happier.

aresant• 16 years ago

In Jason's most recent subscriber email he absolutely THROWS Mark Zuckerberg under the bus for lacking integrity and riding on the edge to grow his business.

If there’s ever been a case of the pot calling the kettle black . . .

tomh-• 16 years ago

I guess as long as you bring in a lot of money for Google they will look the other way. I'm just wondering if their anti spam team is aware of articles like this and why they don't respond? This is really a clear case of spam and its going on for ages..

euroclydon• 16 years ago

Malhalo is starting to remind me of that (was it a Canadian) company who was making all that money off of pay-per-click arbitrage between to search/ad networks, and then one day, Google cut them off cold turkey. Who were they?

OmarIsmail• 16 years ago

GeoSign.

This is the thing. The huge spotlight that guys like Aaron Wall are pointing at Mahalo right now I'm sure is making Calacanis and Co sweating bullets. For big cases like this, it's often a matter of managing perceptions and Google may definitely take action if the story becomes big enough.

The thing is, there are quite a few sites that really skirt the edge of what is acceptable. When these sites/companies are somewhat larger with lots of employees and contractors, well the effect of 'blacklisting' can affect A LOT of people's lives in a very bad way.

For the independent blackhatter, some spam side-site they spin off isn't their livelihood and won't ruin them if banned. Mahalo (and ilk) isn't just a technical matter. Of course if this story gets bigger and Google doesn't do something about it, then this sets a bad precedent for the spam-community to try and game Google. Sure Google will figure ways to deal with it, but if you're the guy dealing with the spam... well, you'd just rather it not exist from the beginning.

I'm really interested to see how this all plays out. From Google's side, Mahalo's side, etc. It's riveting stuff actually.

dustingetz• 16 years ago

he's acting in his incentive, who the hell cares. read: i don't understand why anybody cares.

shareme4• 16 years ago

I wonder what would happen if Jason just tuned off noindex tags on odd days? Clearly turning it off and on gets more blogs talking about it each time

epi0Bauqu3• 16 years ago

FWIW, these pages have been removed from Duck Duck Go for quite some time.

icey2• 16 years ago

Did you have to manually cull results from Mahalo, or did your algorithm discard them automatically?

epi0Bauqu3• 16 years ago

Some automatic, but to be safe I put a manual block on them, which I do for big "useless" sites I keep (or users keep) seeing again and again.

greenlblue• 16 years ago

Can I ask why everyone cares so much about Mahalo? Up to a week ago I didn't even know "human powered search" existed and the consensus seems to be it's an epic fail.

supadog• 16 years ago

Froo is right, there is only one reason why Jason currently gets away with spamming google and that is becuase both google and Mahalo are Sequoia funded. Someone senior at Sequoia has told someone senior at google that mahalo will clean their act up so please dont ban them yet.

froo4• 16 years ago

It's the SEO equivalent of a shell game

Mahalo's disappearing noindex tags

Related Stories