Bit.ly is Harmful to Your Reputation

http://cranialsoup.blogspot.com/2010/04/bitly-is-harmful-to-your-reputation.html

by barrkel • 16 years ago

193 112 16 years ago

cranialsoup.blogspot.com

DenisM2• 16 years ago

What do you think happens when you click it and report a mistake? Do they check the link and remove the flag if the site is ok?

No, they don't. They told me to make a new bit.ly link and give it out to people, as if that would undo the damage that was done

Uhm. Reminds me of Yelp.

jrockway6• 16 years ago

What Twitter needs is to remove the 140 character limitation. Nobody uses SMS anymore, especially when the content contains URLs to non-mobile-friendly sites.

markbao3• 16 years ago

I think that would completely go against the concept of Twitter: short-form communication.

alextgordon2• 16 years ago

They could detect URLs and not count them towards the total message length.

jbellis• 16 years ago

Allowing your users to determine how short is short enough would be a good thing.

patio11• 16 years ago

Is "the concept of Twitter" what people are getting value from? If not, forget "the concept". (In my circles, people seem to be getting value from "an opt-in audience of likeminded people who you may extract business value from someday which does not include your girlfriend, high school buddy, mother, or Farmville." People definitely are not using Twitter primarily for short-form communication: if they were, two tweets out of every three wouldn't include a link to long-form communication.)

ja2ke• 16 years ago

I think Twitter officially supporting and converting URLs into a token at the time a tweet was written would do the trick fine. The 140 limit is what made them, but URLs are obviously a point of failure and frustration, hence the near constant hulabaloo over shorteners and redirects etc. It'd surprising to me that Twitter hasn't offered an official solution for URLs yet, to be honest. They saw people were using @ to reply, and supported it. They saw hash tags and retweets show up and supported them, too. But most users go to a third party to handle links.

jlees3• 16 years ago

yeah, I'm very surprised there isn't a twt.tr or something. Maybe they're holding out for .co...

alexro• 16 years ago

Twitter in fact sort of remove that limit. There is a change called annotations around the corner

philwelch• 16 years ago

Actually I remember a huge discussion a few days ago about how Twitter has a very large base of urban African-American users who do, in fact, use SMS to access it.

justinph• 16 years ago

Tons and tons of people still use twitter via SMS. @EV talked about this during his (otherwise boring) SXSW talk. I don't see them ditching that anytime soon.

jrockway6• 16 years ago

Right, but those people aren't sending URLs to each other.

edanm• 16 years ago

I think Twitter should just not count URL's towards the 140 characters. All URL's should count as one character.

romland2• 16 years ago

How long before all tweets are in fact bogus links, you think?

ericz• 16 years ago

That seems rather stupid. How is url-shortening a already url-shortened url a security threat?

whughes• 16 years ago

Extra cloaking for malicious links? Bit.ly can't exactly check the original site to see if it's OK if it's hidden behind a funky redirection layer. I think that bit.ly was reasonable to have an interstitial, although maybe they should be more open about why and how to remove it.

tkaemming3• 16 years ago

They could check the endpoint URL by following the HTTP headers on 301/302 redirects.

sjs2• 16 years ago

It's a standard http 301 redirect, not a "funky redirection layer".

    /Users/sjs % curl -i http://xrl.in/33qj
    HTTP/1.1 301 Permanent Redirect
    Date: Sun, 09 May 2010 00:38:19 GMT
    Server: Apache
    Location: http://www.donationcoder.com/CodingSnacks/index.php
    Content-Length: 0
    Content-Type: text/html

tkaemming3• 16 years ago

While the author has a right to be upset, her attitude toward the support staff — who likely have no control over the situation — is disgusting. ("Your apology is worthless", "Your refusal to check the link", etc. from the email conversation, and the ensuing comments on the blog post.)

markbao3• 16 years ago

Seriously. I've found it's easier to make someone do something when you don't have your teeth clenched at first, looking to attack. If she approached that with a "Could you help me figure out..." or "I think you should probably fix..." attitude, instead of a "screw you" one, she probably would have received a similarly gracious response.

blahedo• 16 years ago

Did you read the email exchange she posted? Her original email was polite, her first response was a little clipped but not yet angry, and it wasn't until bit.ly had blown her off twice that she started with the attacky language.

csallen• 16 years ago

I see your point. I don't think she's railing against the support staffers personally, but against the company itself. "Your apology is worthless", as in "Bit.ly, your apology is worthless". Still, she could've been a bit more tactful.

Anyway, this is quite an eye-opening article. What bit.ly is doing is a great example of how not to run a business. We should be educating people about the internet, not taking advantage of their ignorance so we can lie about our competitors.

jlees3• 16 years ago

She only started to get uppity after the extremely terse and unforgiving reply from bit.ly ("We don't allow that"), obviously we should all wait a while before writing emails or blog posts while angry, but we're also all human.

nollidge5• 16 years ago

Well, their apology was completely worthless (as they took no substantial corrective action), and they apparently do refuse to check the link.

What's wrong with saying exactly what you mean?

wyclif• 16 years ago

She forgot #3: Just use one of the whitelisted shorteners she mentioned further up in the article, e.g. goo.gl.

jlees3• 16 years ago

Not quite as simple as that, none of the three are readily available to use by just, say, visiting the site and typing in an URL. You need toolbars/extensions/clients etc. If she usually uses a client that's set to xrl.in changing to another shortening service might be a massive pain.

vaporstun• 16 years ago

http://goo.gl/

"Google URL Shortener is currently available for Google products and not for broader consumer use."

DanielRibeiro• 16 years ago

It is interesting to note that common users can easily use goo.gl by using google toolbar's share function.

MWinther4• 16 years ago

How about putting some of the blame on Tweetdeck? Shouldn't "short enough" links be allowed to stay intact?

jackowayed• 16 years ago

Yes, they should. But what's worse? Software that people don't even pay for being stupid in a fairly minor way, or bit.ly being anticompetitive and severely hurting its users in the process? He was focusing on the major issue.

MWinther4• 16 years ago

I agree that bit.ly aren't being cool about this, clearly they should fix it.

On the other hand, you could say that Tweetdeck is actually being stupid in a pretty major way. I have a hard time imagining this scenario occuring without the help of Tweetdeck, or other lazily written convenience shortener functions.

marcusbooster• 16 years ago

Obviously the two companies are working together on this. http://www.nytimes.com/2010/05/03/technology/start-ups/03bet...

nollidge5• 16 years ago

Sure, but shouldn't bit.ly just do the same check?

Osmose• 16 years ago

I'm having a really hard time believing the scenario where a user follows a shortened link in a re-tweet, sees the bit.ly warming page, assumes that the original author is trying to spread malicious stuff (and that the re-tweeter, whom this user is following and supposedly trusts, is supporting this malicious stuff), and starts telling other people about how untrustworthy the original poster is.

In reality, they'd probably close the page, maybe make a reply tweet confused about the warning, and go on with their life.

moultano• 16 years ago

So their spam detection system isn't as good as at should be. Give them a break. People spreading malware use url shorteners all the time, and I'd bet some automated system just got tripped up here.

(malice -> incompetence etc.)

I'm happy bitly is at least working on spam.

tdavis• 16 years ago

Firstly, the hypocrisy of a URL shortener blocking another URL shortener's links because they can't be verified is hilarious. Secondly, if a person goes as far as to shorten every link they copy from their browser and use a Twitter client that shortens all links regardless of length, I hardly think they have much room to complain about the system they're fully backing.

What did she expect to happen? That Bitly would concern themselves with her reputation? If you want a URL shortener who cares, your only option is to run your own. Bitly's only concern is for their business, nothing more.

toni• 16 years ago

In this case, even running your own URL shortener will not help in any way. Bit.ly blocks any already-shortened URL. Whether it's xrl.in, tinyurl or your own.

You can only control your own shortened URL but you have no control when people re-tweet it and pass it through Bit.ly. Which then flags it as harmful.

trezor• 16 years ago

You can only control your own shortened URL but you have no control when people re-tweet it and pass it through Bit.ly. Which then flags it as harmful.

So bitly, a URL-shortening service, is blocking other shortened URLs because they deem shortened URLs harmful? And here I thought the stupidity of twitter and it's ecosystem of useless URL-shorteners had an actual limit.

Confusion• 16 years ago

The blogger reports that bit.ly updated their page, to explain why a link may be disallowed. He doesn't accept their explanation, but it does make a lot of sense:

  - Some URL-shorteners re-use their links, so bit.ly can't
    guarantee the validity of this link.
  - Some URL-shorteners allow their links to be edited, 
    so bit.ly can't tell where this link will lead you.
  - Spam and malware is very often propagated by exploiting
    these loopholes, neither of which bit.ly allows for

amalcon• 16 years ago

It seems a little shaky to me, because: 1) bit.ly could reject the link at submission time, instead of at the time of usage. This would eliminate the whole problem. 2) bit.ly could follow the link, unshorten it, and then shorten the original link. This would eliminate these concerns.

dasil0033• 16 years ago

It makes some shallow sense, but on deeper investigation it's anti-competitive bullshit that is not much better than the scams they claim to be protecting you against.

Think about it, since when is bit.ly the arbiter of link safety? Does bit.ly provide you a guarantee that they will not ever link to malware? Of course not! They could not afford the liability on that. Anyone could post a malicious link anywhere. Any page can set up a 301 redirect any time. People can post malware links anywhere any time.

What's going on here is that bit.ly is breaking links arbitrarily, and doing it to third parties without their consent or even knowledge. It might be possible that a credible legal case could be mounted against bit.ly on this basis. Quite simply, it is neither their right nor their responsibility to be the link police of the internet, and it also is far from within their power. These guys are overstepping in a major way.

nollidge5• 16 years ago

> if a person goes as far as to shorten every link they copy from their browser and use a Twitter client that shortens all links regardless of length

This is not what happened. One person shortened the link from their browser, and tweeted that link. A second person re-tweeted that link, and their Twitter client re-shortened the already-shortened URL.

ck2• 16 years ago

So? Run your own blog with your own rss feeds and stop using twitter.

malnourish2• 16 years ago

Not very feasible for the most part. I don't use twitter, but it does have a large following. This is somewhat along the lines of saying "So? Stop using facebook", although I understand where you're coming from.

nhebb• 16 years ago

Does SMS have to display the href? Could Twitter just replace all URL's with the word 'link' and show the real URL on hover, thereby doing away with URL shorteners altogether?

MWinther4• 16 years ago

Hover doesn't work well with touch based interaction, which is all the rave these days, I hear. I like the idea of showing them separately somehow, though.

pyre4• 16 years ago

It only has to display 'link' in SMS. I don't necessarily think that it has to still say 'link' outside of SMS. It makes more sense to just strip the links when going through the SMS gateway, and count all URLs as 4 characters in the '140 characters' limit.

steveklabnik• 16 years ago

I've typed many a bit.ly link into a web browser directly from my phone. It's easy to do because they're so short.

jrockway6• 16 years ago

But not as easy as finding the tweet in your twitter client and just clicking.

nostrademons• 16 years ago

I solve this problem by not clicking on shortened URLs. At all. If you want me to click on something, send me the full link. If it's too long for Twitter, find me on GChat or send me an email.

toddml4• 16 years ago

Todd from bit.ly here.

From day one, we've prized security, transparency, reliability, and openness at bit.ly. Along the way, we've made a number of product decisions based on those tenets. Among those are link permanence (link destinations don't change once created), the avoidance of anything that interferes with user experience (we've never framed, nor will we), and a dedicated focus on spam and malware detection, so that our users can click on bit.ly links with a high degree of confidence.

We take our responsibility as internet citizens seriously, and you'll see this exhibited even in the small details of the ways in which we manage flagged links (you'll notice we never actually disable a redirect, and at most simply insert an interstitial which retains the end destination link).

In the course of analyzing content for spam, malware, and phishing attacks, we rely on a number of systems, both internal and external. Over the course of the past year, a number of spammers have attempted to use various levels of indirection through redirectors (some of which are reconfigurable), in order to obfuscate and cloak their efforts. In fact, the bulk of shortens to bit.ly coming through other URL shorteners have tended to be attempts to spam the system. While our crawlers do of course follow links through redirections, the inclusion of modifiable redirects in the stream, and our analysis of the preponderance of spam attempts via these vectors have made it necessary and appropriate in some cases to block the URL shorteners.

Just to reiterate, the only goal is and always has been to protect the end user clicking on bit.ly links, regardless of the link source. Given that multiple layer wrapping of URL redirectors tends to be an edge case based on inappropriate API usage, confused users, or in the preponderance of cases, attempts to spam, we think this has been a fair approach. As such, you'll note that we did in fact update our interstitial warning pages with language better reflecting the reasoning behind the status. We're happy to see a healthy, vibrant, shortening ecosystem, and have no intention whatsoever to put a damper on other sites in the space.

Some have suggested we simply not shorten URLs already pointing to 3rd party short URLs. While this is a potential possibility, our API responses and the innumerable clients and scripts that use these methods aren't currently designed with this state in mind. Consequently, any changes would have to be carefully considered.

As with any product, bit.ly is a work in progress, and we're always interested in finding ways to best serve our users, while maintaining the integrity and openness of the product.

tyler2• 16 years ago

You say "any changes would have to be carefully considered". So, are said changes being considered or are they not?

toddml4• 16 years ago

We always consider ways in which to provide the best level of service to our end users and API users, while preventing unintended side effects. There are pros and cons to every approach, and they have to be evaluated with care.

spohlenz2• 16 years ago

> While our crawlers do of course follow links through redirections, the inclusion of modifiable redirects in the stream, and our analysis of the preponderance of spam attempts via these vectors have made it necessary and appropriate in some cases to block the URL shorteners.

Why not do as tkaemming suggests and follow the redirections to link to the final endpoint URL?

toddml4• 16 years ago

The intent behind the very statement you quoted was to convey that we do precisely that. However, also mentioned was the fact that in a number of cases, modifiable destination redirects are embedded within the chain. In those cases, unless the redirect is crawled on every clickthrough, the integrity of the chain is difficult to assert.

nexneo• 16 years ago

bit.ly is not doing right. cut the crap, fix it.

rapind• 16 years ago

Seems reasonable to me. Just out of curiosity since this article is placing the blame at bitly's feet and not tweetdeck, how do the other shortener services behave given a simarly shortened URL. Are they whitelisting?

Lol_Lolovici• 16 years ago

> our API responses and the innumerable clients and scripts that use these methods aren't currently designed with this state in mind

Well, you could just return the original URL when it is already shortened and this way there is no new "state" to worry about.

PanMan• 16 years ago

Offcouse that IS a new state. If I call bit.ly's shorten function, I expect a bit.ly link back, not the original URL. I can imagine, for example, not saving the bit.ly part in a db, only the identifier. That breaks (without proper checking) if the original URL is returned.

dasil0033• 16 years ago

That's well-written PR Todd, but the bottom line is this: your false positives are doing potentially severe damage to third parties. These are people who aren't even using your service.

When these things happen you need a way to fix them quickly, or you will find yourselves in legal hot water sooner or later.

Also, I might suggest to you that you have neither the power nor the authority to be the link police on the internet. What you're doing is engaging in an arms race that A) is impossible to win, and B) has numerous innocent bystanders.

I've never really been one to decry the dangers of link shorteners, but this is a great example of how a link shortener—even with a team of stand-up ethical guys behind it—can be bad for the internet.

We've been thinking about signing up for bit.ly pro, and I have to say this throws a wet blanket over the whole thing.

DenisM2• 16 years ago

We've been thinking about signing up for bit.ly pro, and I have to say this throws a wet blanket over the whole thing.

whitelabel shouldn't be nearly as bad - as long as you control the domain name you can always take your ball and go home.

qeorge• 16 years ago

I have to agree with the author, this is ridiculous. They even do this to tinyurl:

http://bit.ly/cGLU0o

Tinyurl was without a doubt bit.ly's #1 competitor, especially when their service launched, and they know you can't edit a tinyurl. Seems like a pretty convenient time for an URL shortener to suddenly get preachy about the dangers of URL shorteners.

caffo• 16 years ago

I've contacted bit.ly about this issue 4 months ago. Got the same answer - "Page is under review". Maybe its an indirect attack to the competitors?

malnourish2• 16 years ago

Unfortunately, offering itself as the best alternative to protecting links when setting a warning page for already shortened links doesn't seem very "indirect".

bcl• 16 years ago

Gee, how about we stop acting like we need to limit things to 140 characters? This isn't SMS you know. We do have the bandwidth to actually exchange useful information and real URLs. Twitter, bit.ly, et. al. are all pretty damn silly.

Even back in the days of 300bd modems and BBS's we weren't this dumb.

nollidge5• 16 years ago

The brevity is what makes Twitter distinct from everything else.

carussell4• 16 years ago

Funny, I thought the reason people should use a service is its worth, not auxiliary novelties.

If I make a pitch to somebody, do you think I should rest assured that they'll go for it if I word it in the following way, "Now I know that by almost all metrics this kind of behavior is bad, but we did some looking into what other people are doing, and it's unique."?

vsync2• 16 years ago

Twitter is still fully usable via SMS and that's both useful and unique.

carussell4• 16 years ago

Are there any handsets in use out there that support SMS and don't support Concatenated SMS? This is a serious question.

BrandonM2• 16 years ago

This isn't SMS you know.

But isn't it? I have several friends who get their Twitter updates via SMS.

Hume62• 16 years ago

Isn't the problem bigger than a small number of people not paying attention to their Twitter client re-shortening links when it's not necessary? Aren't people intentionally re-shortening links they get from others, so that they can track their own reach/influence? If I have a big following and I re-shorten your link, I can then go to Bit.ly or (anywhere else I have an account) and exactly measure how that link was spread across the net, regardless of whether any RTs mention me. "I'll take your short link and raise you my own..."

gsiener• 16 years ago

My takeaway is to make sure I'm using the most popular shortener (aka bit.ly)

Bit.ly is Harmful to Your Reputation

Related Stories