Why Twitter's Engineers Hate the @replies feature

http://www.25hoursaday.com/weblog/2009/05/15/WhyTwittersEngineersHateTheRepliesFeature.aspx

by Anon84 • 17 years ago

30 37 17 years ago

nop• 17 years ago

"What have I missed here?"

They don't have to check 1.7 million users to see if they have @replies on, if they maintain a list with "@reply subscribers", there's little overhead as far as I can see. If a something rarely changes but is checked a lot, it's a pretty good idea to have it pre-calculated.

Edited: "No overhead", was a bit to absolute thanks jpwagner.

Re-Edited: Oh look there was a commenting feature on the blog, was pretty hard to find.

jpwagner2• 17 years ago

agreed. actually that's not NO overhead, but it's trivial and the right solution.

instead of just one flag link on an HN post, there should be

flag as inappropriate vs.

flag as stupid

*edit: make that "flag as waste of time"

Carnage4Life2• 17 years ago

That solution doesn't address my question. Caching @reply subscribers only addresses the 3% of users who have opted-in to receiving all @replies. For the remaining 97% whether they receive @replies from someone they are following or not is a function of which user the reply is directed to and whether that user is also their friend. You can't cache that, at best you can optimize how you calculate who should receive replies.

In fact, what you've pointed out is that building an implementation to address the 3% case is straightforward which was the point of my post.

PS: My blog does have a commenting feature. In fact there are 3 comments in response to the post.

joshwa• 17 years ago

those comments are very well hidden! Why not just display them by default they way the other 99% of blogs do?

Or if you're going to hide them, at least put the view comments link at the bottom of the post where I'd expect the comments to be.

jpwagner2• 17 years ago

Person X member of 97%

Person Y member of 97%

Person Z member of 3%

Person W member of 3%

________________________

Person_A_@_List: X, Y, Z

Person_B_@_List: X, W

________________________

Person_A writes: @Person_B you be cool!

Received by X and Z...not Y or W

ck113• 17 years ago

If you're interested in wild speculation (I have no idea how Twitter's database works), here's how I interpreted Biz's explanation:

With the new system, say user Foo writes "@Bar lol me too!". Then Twitter can take Foo's follower list, join it with Bar's follower list, and send the message to everyone in the resulting list. Relational databases are very good at joins.

On the other hand, with the old system, they'd have to do a deep inspection of the record for each of Foo's followers to know if they should send the message to that follower. Relational databases are much less good at this.

But, as you and others have pointed out, if the number of users that use the "all @-replies" feature is really so small, it would be fairly inexpensive to cache the list of all of Foo's followers who use that feature, and join them in as well. I don't know why they don't do that -- maybe it adds up (like, if only 3% of users use the feature, but those users follow a lot of other users, they'll each end up in a lot of other users' caches).

dws• 17 years ago

The 3% (of users affected) number has been trotted out a few times. Has twitter ever backed that up? It feels like a way of marginalizing the people who're complaining about the change, by painting them (us) as a vocal minority.

(Following up: A trusted source confirms 3%)

gaius5• 17 years ago

The thing you're missing is that everyone at Twitter slept through CS 101.

dasil0033• 17 years ago

Give me a break, scaling Twitter is not easy. CS principles do not magically allow you to construct an optimal solution for real world engineering problems. At Twitter's volume, the tiniest bit of extra latency in some middleware or network hardware can be the bottleneck.

It's so easy to sit here and play armchair engineer and diagnose Twitter's (mostly past, btw) problems with a nauseating mix of hubris and ignorance.

cedsav• 17 years ago

What if they have a fast way to get the intersect between followers of 2 different users? In this case, if a message is a @reply they could ignore the follower lists of each users and use the intersect list only (which can be cached). Supporting the 'get all replies' option means that they also need to consider the full follower lists and then deduplicate. .. just guessing here

Tichy• 17 years ago

It really does not make any sense to me, given that @replies were not even a feature of twitter originally. Isn't the folk story that they only made @replies into a link when all people started marking replies in such a way?

The way it is now (from my understanding) is that for every reply they have to check if somebody is following the person that is being replied to. How can this be more efficient than simply sending the reply-tweets to all followers (which they have to do anyway with all the other tweets that are not replies)?

zimbabwe• 17 years ago

Isn't the folk story that they only made @replies into a link when all people started marking replies in such a way?

Considering they've just recently started turning #hashtags into links, I would not at all be surprised if @replies were birthed in a similar manner.

gojomo• 17 years ago

First, I sympathize with the Twitter engineers who have had to build a scaling system improvisationally under incredible growth and time constraints. I doubt they have time to reevaluate early limiting decisions except when things are on fire, and even then they probably only have enough time to go for the quick fix.

Ultimately, though, I'm sure they'll figure out a way for this to work -- especially as it only requires "3%" of users to get @unacquiantanted-replies the same way they get broadcast messages.

One approach: split accounts into two inner accounts, for example, aplusk and aplusk(all). Only the original name is ever publicly shown -- though the account's public page shows the all tweets, as now. However, which one people actually 'follow' depends on their 'show all @unacquianted replies' setting. Thus every aplusk(all) tweet is handled as a broadcast, because all those followers -- only 3% of the total -- prefer it that way.

varenc• 17 years ago

"In that case I’d expect Twitter to argue that the feature they want to remove for engineering reasons is filtering out some of the tweets you see based on whether you are a follower of the person the message is directed to not the other way around. "

That would be easier for the twitter engineers, but thats not what 97% of twitter users want. The author doesn't acknowledge this.

jpwatts• 17 years ago

Lies, damned lies, and statistics.

That 97% only indicates that most people don't change the defaults; it says nothing about what Twitter users want.

In this case it's particularly misleading, since we're talking about an option to turn off silent data loss. What would make someone track down a setting related to a problem they don't know they have? A more honest representation would be: "A small number of our users were aware that we filter @replies by default, some were happy about it and a few (3% of total users) changed their settings to remove the filtering."

SMrF• 17 years ago

Is anyone else getting redirected from the article to rotten.com?

aardvark• 17 years ago

All I'm getting is "service unavailable"

Why Twitter's Engineers Hate the @replies feature

Related Stories