Twitter's UID associated to each tweet will very soon exceed 2,147,483,647

http://www.twitpocalypse.com/

by timothychung • 17 years ago

66 43 17 years ago

twitpocalypse.com

cl3m• 17 years ago

More at http://blog.programmableweb.com/2009/06/09/the-twitpocalypse...

sho2• 17 years ago

That is quite an amazingly high number. Surely 100m people have not posted 2,000 "tweets" each.

I wonder what proportion of the total is human-typed, and how much is machine generated?

axod• 17 years ago

Amazingly high? considering how long twitter has been going? It's extremely small IMHO.

http://news.ycombinator.com/item?id=648725

"Just 10% of Twitter users generate more than 90% of the content, a Harvard study of 300,000 users found."

I'd expect most of it is machine generated.

hugothefrog2• 17 years ago

I'd also expect most of those numbers not to be used. Their db writes are probably not contending on one monotonic counter. They're probably using some numbering scheme which guarantees uniqueness across multiple writable databases, but not global serialisation based on id.

cperciva• 17 years ago

Surely 100m people have not posted 2,000 "tweets" each.

Two billion tweets would be 1M people posting 2k tweets each, or 100M people posting 20 tweets each.

sho2• 17 years ago

Oh. Oops. Of course you are right. That was pretty dumb of me : /

pierrefar2• 17 years ago

It's actually sooner. They assume 83 tweets per second (reference 2) but the number is 200 tweets per second: http://friendfeed.com/scobleizer/50e673d8/some-stats-from-tw...

m_eiman2• 17 years ago

From the link:

"Twitter is seeing about 200 tweets per second, during peak loads. - Robert Scoble"

Note "during peak loads", so 83 is probably not too far off.

pierrefar2• 17 years ago

Fair enough. Do we have any solid references of the actual average rate of tweets?

jsdalton• 17 years ago

They must read HN. It now says:

2 at a rate of 151 tweets per seconds

IsaacL• 17 years ago

"Values updated every 5 minutes". Looking at the source (I don't actually know JavaScript, but it seems pretty simple) I'm assuming that means they get the current TwitID AND the current rate of increase.

DrJokepu• 17 years ago

In the age of multicore 64 bit CPUs, they're not using signed 32 bit integers as primary key, do they?

domdefelice• 17 years ago

I think the problem is not in twitter itself but in the services using their APIs.

thristian• 17 years ago

To be fair, they say "For some of your favorite third-party Twitter services not designed to handle such a case..." rather than accusing Twitter themselves of primary-key ineptitude.

ks• 17 years ago

Twitter is using an unsigned 64-bit bigint according to this:

http://twitter.com/twitterapi/status/2048659057

Steve0• 17 years ago

Meh, anyone who's coding for this platform must have put those id's in a unsigned integer, wouldn't they?

tjogin• 17 years ago

It's such a low barrier of entry and almost non-existant learning curve to code for Twitter, so I expect quite a lot of services to have made that mistake. Not the big ones, though.

patio11• 17 years ago

I suspect for a lot of people coding against Twitter the notion of integers being signed or unsigned is about as foreign to their experience as pointer arithmetic. I do not consider that a bad thing.

irb > "2,147,483,647".gsub(",","").to_i + 1

2147483648

(Forgive the extra code necessary to strip out the commas. The Windows irb console leaves a little to be desired in terms of midline editing.)

there• 17 years ago

you know ruby supports _ as a comma separator, right?

irb(main):002:0> 2_147_483_647 + 1

=> 2147483648

TweedHeads2• 17 years ago

ALTER TABLE twits MODIFY COLUMN twitid BIGINT UNSIGNED NOT NULL

There, fixed, send check for $20 to donations@redcross.org

Call me back when they reach 18446744073709551615

sanj• 17 years ago

Assuming that Rails is still interacting with the DB, and the ID in question is the primary key (ie, the 'id' on the 'twits' table) isn't quite that easy.

I had to do this a couple of years back when I realized I had a table that'd grow quickly:

http://snippets.dzone.com/posts/show/4422

Feel free to send the donation to the same place.

lpgauth• 17 years ago

Is there no easier way then this? Can't believe that wasn't integrated into rails...

fleaflicker• 17 years ago

That command will take several hours to complete and write lock the table.

TweedHeads2• 17 years ago

Nothing the fail whale can't fix ;-)

jsonscripter2• 17 years ago

Why can't you just redirect the writes to a new temporary table and join the original and temporary on reads? (I'm not a database admin)

Andys• 17 years ago

Unless you use PostgreSQL.

But then you would have had 64 bit ID by default anyway.

joezydeco• 17 years ago

Hmm, didn't fix my iPhone client. Try again!

lallysingh• 17 years ago

Anyone notice the footnote on the estimated time of crossing?

"3 the average tweets per second and the Twitpocalypse date are a rough approximation, please do not schedule any vacation around that date"

ivankirigin• 17 years ago

Maybe I'm just paranoid, but I index the tweet id's in Tipjoy with a string: tweetId = models.CharField(max_length=50, db_index=True)

dpcan• 17 years ago

I figure many twitter apps are written with SQLite as their database (in AIR, etc) so the Integer, Primary Key, can get up to 9223372036854775807

http://www.sqlite.org/faq.html

What is the point of creating a website like this?

drawkbox• 17 years ago

Always use a UUID or GUID (same thing) in todays web applications. When we get to terrabytes of data as the norm for apps int based keys are going to seem so ancient.

Sure it takes more room but space it cheap, and it makes syncing and distributed databases so much easier (not tied to one machine or cluster)

charlesju• 17 years ago

I'm going to put money down that this will not affect anything.

Twitter's UID associated to each tweet will very soon exceed 2,147,483,647

Related Stories