Chess.com stopped working on 32bit iPads because 2^31 games have been played

https://www.chess.com/forum/view/general/impossible-de-jouer-depuis-deux-jours-quel-est-le-probleme

by NewGier • 9 years ago

757 326 9 years ago

chess.com

prh8• 9 years ago

Real world example of why Apple is killing 32 bit apps on iOS.

dfox2• 9 years ago

This has nothing to do with CPU architecture.

mikeash8• 9 years ago

It's a little related. The languages typically used for iOS programming encourage the use of data types whose size matches the CPU architecture's bitness. Thus, careless programmers will end up using 32-bit integer types on 32-bit devices, and 64-bit types on 64-bit devices.

I really doubt this is in any way linked to Apple's reasons for dropping 32-bit, though.

_puk• 9 years ago

"The reason that some iOS devices are unable to connect to live chess games is because of a limit in 32bit devices which cannot handle gameIDs above 2,147,483,647."

BinaryIdiot• 9 years ago

This is simply because they're taking an integer from their database as an auto incrementing ID and converting it directly into a native integer on the iOS device thus breaking it. They could work around this any number of ways.

It's a pretty lame bug, to be honest and certainly something easily foreseeable as this wasn't an overnight occurrence.

remus• 9 years ago

How so? A 32 bit platform doesn't mean you're limited to 32 bit ints....

vxxzy• 9 years ago

How many other examples like this have occurred throughout computing history?

SamReidHughes3• 9 years ago

Plenty, Twitter for example.

dfox2• 9 years ago

IIRC at least slashdot and twitter had to do some non trivial database migrations because they hit maximum value of some ID field.

mikeash8• 9 years ago

Twitter's was called the twitpocalypse and it was a pretty big deal at the time. I don't know how bad it was internally, but it hit a lot of third-party apps that stored tweet IDs in 32-bit integers.

One interesting aspect of it was that Twitter realized what was going to happen in advance, and artificially pushed their IDs over the edge at a preplanned time so they could have as many people available as possible to work on any problems that appeared.

cm21872• 9 years ago

Well, famously youtube's views counter overflow from Gangnam Style...

0003• 9 years ago

That was a joke though and they were prepared.

nisse72• 9 years ago

IP4 address space comes to mind.

And I think it isn't uncommon to have to add digits to the front of telephone numbers as regions grow (or telephones become more common) and the number space isn't large enough.

creepydata3• 9 years ago

I mentioned in in another commend but Pacman crashes at level 256

http://pacman.wikia.com/wiki/Map_256_Glitch

Slartie• 9 years ago

World of Warcraft for quite some time, I think about 4 years, had a "mysterious" limit for the maximum amount of gold that you could accumulate on a single character. It was 214.748 gold, 36 silver and 47 copper...

At least they put an actual check in there - you didn't suddenly overrun and wake up with an enormous debt, so I'll give them some credit for that ;-)

rsynnott2• 9 years ago

A particularly expensive example: https://en.m.wikipedia.org/wiki/Cluster_(spacecraft)#Launch_...

eicossa• 9 years ago

IIRC the whole "Gandhi is gonna nuke the rest of the world" meme came from such a bug occuring in the world-domination strategy game Civilization 2.

Bakary2• 9 years ago

They had meant to give him the lowest aggression rating possible, but accidentally inputted -1, which then looped back the other way to the highest rating possible. Nukes soon followed.

matheweis• 9 years ago

Microsoft Lync has an as-yet unfixed bug where after 49.7103 days (4294967294 aka 2^32 milliseconds) of system uptime, it locks the Lync status indicator to "away". The status indicator is otherwise unusable thereafter until a full system reboot.

See: https://social.technet.microsoft.com/Forums/en-US/1e76e106-4...

masklinn2• 9 years ago

A counter overflow factored in the THERAC-25's race condition (one of the software's interlock was an overflowable counter rather than a flag, if the operator started treatment right as the counter overflowed the system would proceed rather than refuse).

SomeHacker44• 9 years ago

"This was obviously an unforeseen bug that was nearly impossible to anticipate..."

Snarky... Except that there were probably years of games to notice that you were approaching a "magic number" like 2^31.

CGamesPlay• 9 years ago

I actually read that quote as fully sarcastic.

blktiger• 9 years ago

As expected, sarcasm always translates correctly into textual form.

comboy• 9 years ago

Poe's law.

blacksqr• 9 years ago

On the contrary, this is pretty much the perfect example of the completely anticipatible bug.

floatingatoll• 9 years ago

That's not snark. That's just self-doubt, expressed. "We never thought we'd host 2^31 chess games."

EDIT: Apparently they already said that: https://news.ycombinator.com/item?id=14540509

rasz• 9 years ago

were they ever expecting negative number of games? why signed integer?

SamReidHughes3• 9 years ago

It's very reasonable. This way they overflow into invalid values instead of zero.

MereInterest• 9 years ago

Assuming you're working in a language that defines signed integer overflow. Depending on the language, you can result in undefined behavior, instead. For that reason, I'd go with an unsigned counter, with the first million IDs being invalid or reserved for future use. That way, you get well-defined overflow into an invalid region.

Inityx• 9 years ago

Because signed is default for some reason in most languages, and most developers aren't taught to think critically about how decisions like simple datatypes might affect scalability.

koolba• 9 years ago

Not just the default, it doesn't even exist in some languages.

libria• 9 years ago

The problem is momentum. I could use unsigned int everywhere, but then I have to constantly typecast to int and back anywhere I use a library expecting signed ints. If we all switched to unsigned int by default, then everything would make more sense but we'll all live in typecasting hell during the migration.

joshhart• 9 years ago

Also note that some languages like Java do not even support unsigned integers.

asimpletune• 9 years ago

I read somewhere in the swift documentation that unless you have a specific need for a UInt, that Int is preferred even if you know that the value will always be nonnegative. I think compatibility is one reason they give.

MichaelBurge• 9 years ago

You need to first establish that the type was chosen intentionally before asking why it was intentionally chosen. Otherwise the question is ill-formed.

It looks like they are using PHP/MySQL/Javascript/Flash, with only MySQL having any explicit types.

Even so, an error is often preferable to overflow, which is usually undefined behavior and could lead to a duplicate primary key anyways if it wraps to the first game.

A better question is "why 32-bit over 64-bit", but the site dates back to 2005 where that was the norm and the question has the same issues.

lallysingh• 9 years ago

Compilers can optimize for signed integers better. Overflow/underflow on signed integers is undefined behavior, which is space for compilers to optimize. But unsigned ints are defined for all cases, so you get less optimal code.

Also, you have problems whenever you compare against signed ints.

cm21872• 9 years ago

Self-confidence as a programmer is when starting a new project, storing the transaction ID as a long rather than an int...

trendia• 9 years ago

Or UUID...

derefr• 9 years ago

I pick UUIDs because experience has taught me that even for the smallest workloads, I'll inevitably have to shard my DB (to partition a shared public cloud from N isolated "enterprise" deployments), and then will inevitably want to do statistical-analysis things that involve ingesting rows from the shards (or log entries referencing those rows), and deduplicating them by ID, without generating false-positive collisions in the process.

The simplest way to do that is to just throw UUID at all problems from the start. (https://github.com/alizain/ulid s are better, but there aren't libraries to generate them in literally every language + RDBMS.)

rdtsc• 9 years ago

Self-confidence as a programmer is when starting a new project, storing the transaction ID as a long rather than an int...

uint64_t even

Or a UUID as others have suggested.

Technically C spec doesn't really say exactly how many bits int, long and long long should be. If you want specific sizes and your code to be somewhat portable use the specific bit sizes to make that clear. There are also types for size-like things (size_t) and pointer and offset like things.

fpgaminer2• 9 years ago

If you want specific sizes

I would go further and say you should _always_ use specific sizes, unless forced otherwise. There's no reason not to.

bitwize• 9 years ago

The C spec does specify minimum sizes: at least 16 bits for int, 32 for long, and 64 for long long.

Best to use the stdint types, just in case.

bcoates• 9 years ago

Your code is actually less portable if you use types like uint64_t -- if your system doesn't have exactly that type implemented the typedef won't exist. If all you need is a really big number 'unsigned long long' is required to exist and be able to store 0..2^64-1

dom02• 9 years ago

That is, unless you're on Windows.

simon83• 9 years ago

The naive, 9 years in the past me once was like "int will last us forever! and it'll save us some space!", only to have to change it to bigint a few months later

WrtCdEvrydy• 9 years ago

Also see: IPv4 vs IPv6.

Remember, these fancy computing devices were built for the rich and the government, not for the average joe, noone thought computing would be this easily accessible.

maxxxxx2• 9 years ago

"int will last us forever! and it'll save us some space!",

There are still plenty of people with that mindset. Some will even quote "YAGNI" when you question them.

maxxxxx2• 9 years ago

It gets better when you realize that on Windows a long is still only 32 bit....

qb45• 9 years ago

You mean 64bit Windows?

It gets better yet when you realize that on 32bit systems (like in TFA) long usually is 32bit too ;)

lacampbell• 9 years ago

You know what, you're right. I'm going to change some SERIALs to BIGSERIAL in the database of my side project. Someone has to start believing in it - it may as well be me.

taneq2• 9 years ago

So when is it self-confidence and when is it overengineering and invoking the wrath of YAGNI? :P

hvidgaard3• 9 years ago

It's the smart thing to do - 4.3 billion is not that much.

I had some students that asked me if even a long would be enough to handle exponential growth, after all it's only twice as big. As a thought experiement I asked them to come up with a time to fill a 32 bit int completely. They came up with roughly a year. Then as a margin of error I said, let's assume you have 4.3 billion transactions every second instead. This volume can be sustained for 100 years, and we're still not in the danger zone yet.

ReverseCold• 9 years ago

2^64-1 is a lot more than 2 times 2^32-1

pc86• 9 years ago

Uh it's not twice as big.

    2^32  ~              4,294,967,000
    2^64  ~ 18,446,700,000,000,000,000

ericfriday• 9 years ago

This reminds me YouTube changed its view counter from 32-bit integers to 64-bit integers due to the popularity of 'Gangnam style' https://www.wired.com/2014/12/gangnam-style-youtube-math/

smitherfield• 9 years ago

That was a joke; it was always a 64-bit integer.

akerro2• 9 years ago

WHAT?

wnoise2• 9 years ago

Do you have a source for that?

devrandomguy• 9 years ago

I visited that video specifically because the view counter was jammed at UINT_MAX. There were comments confirming that everyone was now visitor number 4,294,967,295. In fact, it might have been an HN post that brought it to my attention; I totally didn't get sidetracked on YT and end up watching K-pop all afternoon.

eponeponepon• 9 years ago

It's fascinating... the Y2K problem never came to fruition because - arguably - of the immense effort put in behind the scenes by people who understood what might have happened if they hadn't. The end result has been that the entire class of problems is overlooked, because people see it as having been a fuss over nothing.

I sometimes think it would've been better if a few things had visibly failed in January 2000.

mkempe• 9 years ago

My long-term active retirement plan involves building a year-2038 consultancy.

glhaynes• 9 years ago

Similar to how much effort goes into dealing with things like dangerous strains of bird flu, only to have people complain about how much money was spent on "nothing" when an outbreak doesn't occur.

sbov2• 9 years ago

This is a whole class of problem - I wonder if there's a name for it. More examples include: talking about welfare being unnecessary because no one is starving. Or people on medications stopping because they feel better (while still on them).

rcthompson• 9 years ago

I wonder if there's a general law here, something like "prevention is socially unsustainable".

justin667• 9 years ago

I sometimes think it would've been better if a few things had visibly failed in January 2000.

Having to spend billions of dollars on programmers' goofy means of saving a few bytes of memory is a pretty visible failure.

andrepd2• 9 years ago

How many bytes of memory were saved? 32-bits multiplied by potentially trillions of time, considering storage was not dirt cheap as it is now...

wpietri10• 9 years ago

Nope. Definitely not goofy.

My dad was a programmer in the early days. The machines he started on in the 1960s had 8 KB of RAM. Saving a byte then is the equivalent today of saving 1 MB on an 8 GB machine.

Multiply that times, say, the thousands of customer orders you're trying to process and the goofy thing would be burning a lot of additional RAM because it might help somebody 35 years later. Who among us is writing code today worried about how it will be used in 2052?

Retric• 9 years ago

It really depends on how much data and how old.

My dad's first programming job was initially to mechanically change how variables where stored thus saving 1 and only 1 byte of disk space. Someone ran the numbers and having someone do that for a few months was a net savings.

A few years after that he was talking about some relatively minor optimizations that saved a full million dollars worth of hardware costs by delaying a single new computer purchase.

eveningcoffee• 9 years ago

I never have thought this way. If this is true then there should have been highly visible media campaign to point this out.

zeta0134• 9 years ago

Maybe it's because of how young I was at the time, but I remember media outlets reporting on the Y2K problem, and showing video of hundreds of older computing devices that were being retired (thrown in landfills) because they wouldn't work properly after the date change happened.

Maybe this isn't a "real" failure, and the symptom of some IT departments working diligently to solve the problem before it happened. In any case though, I'm curious how inaccurate the televised reality from my youth actually was.

rovr138• 9 years ago

Y2K was a software issue.

In short and very simple terms, some software stored date as DD/MM/YY.

It would asume that for year it would always prefix 19. The problem was when you reached /00. Any calculations or software decisions that happened based on that date, it would be off. Way off.

Some solutions where expanding the date to YYYY or adding a prefix to the new dates. Dates without the prefix would be 19xx. Date with the prefix X was 19xx and dates with prefix Y where 20xx.

asveikau• 9 years ago

The problem was that media reports at the time seemed to assume that anything with a computer would completely blow up, instead of rationally thinking through much more benign consequences of a date or calendar being wrong.

The image of a bunch of perfectly capable computers being discarded for no reason describes this media frenzy wonderfully.

rabboRubble2• 9 years ago

As a former Y2K project manager, I had one thing go BOOM. But it didn't go boom on 1/1/2000, rather on 2/29/2000. Our Y2k program had, IIRC, some 9 likely fail dates including the 1/1/2000 date. The component in question was provided globally and untestable locally, so I the best I could do is acquire a copy of the remote complete test script and call it good.

I was thrilled that we had something to point to as a "see, this is why we put in so much work". Prior to that had received lots of criticism about the amounts spent, people hired, blah blah blah.

jfultz• 9 years ago

If you were watching closely and knew what to look for in the first couple of months of 2000, the failures were there. But they were generally minor and easy to overlook as Y2K problems.

I spotted something like half a dozen failures in various systems I interact with which I strongly suspected, based upon the timing, were likely Y2K problems that slipped through testing. For example, I received duplicate bills for one of my credit cards for the January 2000 billing period, and then a subsequent apology for the duplicate bills. They never said Y2K, but the timing was very suggestive.

It's pretty much exactly what I expected from most companies...the big stuff had been largely been dealt with, but a few things slipped through which they could dismiss with some hand-waving. The thing that surprised me was that there didn't really seem to be any high profile disasters (like a company that couldn't ship products, an airline that couldn't issue tickets, or whatever) at all...I figured there'd be at least a couple.

collinmanderson• 9 years ago

in JavaScript, new Date().getYear() returns 117 :)

dspillett2• 9 years ago

in the first couple of months of 2000, the failures were there.

They were there before then too. Things that could go wrong at midnight on NYE were only one of a few classes of problem associated with roll-overs. There were a lot of bugs in like scheduling applications (and similar system tools) in the run up to 2000 that the man on the street didn't associate with the Y2K issue because it didn't happen at that exact moment.

moomin• 9 years ago

Having spent a year hardening financial systems against Y2K, I was very unimpressed to discover my credit card not working on 1st of January.

The call centre staff told me that this wasn't a Y2K bug, but a year-end bug. As if that was meant to make me feel better about an obvious, grim, failure.

sundvor3• 9 years ago

"Wait, we spent two million dollars servicing your fleet of 100 cars in the last year, and you're complaining that none of them broke down? Why spend all that money on maintenance when they just keep running?"

Danieru• 9 years ago

Are you sue those numbers are right? That's 20k per car per year. At such a cost running the cars with no maintenance then buying new cars every break down is cheaper.

megablast• 9 years ago

the Y2K problem never came to fruition because - arguably - of the immense effort put in behind the scenes by people who understood what might have happened if they hadn't.

Except many countries who spent no money on upgrading their system has very few problems too.

Piskvorrr5• 9 years ago

[citation-needed]

wmil• 9 years ago

There actually were a lot of inconsequential failures with date display.

A fair number of websites went from showing "Dec. 31, 1999" to showing "Jan. 1, 19100".

weavie• 9 years ago

The National Express (a UK based coach company) website showed the current date as 01/01/19100 on new years day 2000.

throwaway2016a2• 9 years ago

Reminds me of the havoc that was caused when Twitter tweet IDs rolled over. Resulting in every third party developer to update their apps (and at the time there were a lot of those).

Twitter saw it coming and forced the issue. By saying that at a certain date and time they would manually jump the ID numbers rather than wait for it to happen at some unpredictable time.

syncsynchalt3• 9 years ago

They didn't roll over, they exceeded 2^53-1 which is the max Number which doesn't truncate when treated as an integer in js. The solution was to treat it as a string.

(Or we're thinking of different events, I apologize if so)

spullara3• 9 years ago

Yep, it would just point to the wrong tweet. Fun stuff. So many of these time bombs.

gilgoomesh2• 9 years ago

Twitter must have been misleading when they communicated the reasons for this change since they did not exceed 2^53-1, nor do they expect to exceed this in the near future.

From a (former) Twitter dev:

Given the current allocation rate, they'll probably never overflow Javascript's precision nor get anywhere near the 64-bit integer space.

https://twittercommunity.com/t/discussion-for-moving-to-64-b...

throwaway2016a2• 9 years ago

Rollover was a confusing word for me to use. I did not mean it in terms of integer overflow. I meant it in accounting terms. As in to roll from one namespace to another.

I could have definitely chosen my words better.

shurcooL• 9 years ago

Do we know when chess.com launched? If so, we can calculate the average number of games being played per second.

CDRdude• 9 years ago

Wikipedia says "June 2007", which I'll approximate to 10 years. That gives us 6.8 games per second.

jakub_g• 9 years ago

Long long time ago, I created a poll on a small website I was maintaining. I didn't expect much traffic and, so, not thinking too much about it, I put the ID column to be a TINYINT (i.e. max value = 255)...

That was a valuable lesson.

(I actually generated most entries myself while testing stuff - live in prod of course - and while there were probably fewer than 255 votes, the AUTO_INCREMENT did its job and produced an overflow).

ryandrake• 9 years ago

"Long long time ago"

Seems you have learned your lesson :-)

mattkenefick• 9 years ago

"Obviously unforeseen.. impossible to predict." Really? You don't know how to properly store ID numbers?

IMPOSSIBLE to predict.

phailhaus• 9 years ago

That's clearly sarcasm.

phonon• 9 years ago

And I was just reading Heroku/Django discussing the same issue this morning!

https://groups.google.com/forum/m/#!topic/django-developers/...

abalone2• 9 years ago

So they probably just need to use longs instead of ints. But I'm curious, if you were really stuck with a 32-bit limit on data types, what's your preferred workaround? I'm thinking I'd add another field that represents a partition. Are there other "tricks"?

reformatt• 9 years ago

If you could only use 32-bit data types, you can get 64-bits from using 2 integers together like a long number. So the right integer would hit the max, start over at 0, and increment the left integer. Then, using this idea you can create a class of numbers that can have however many bits you want by using more ints.

abalone2• 9 years ago

Cool, yeah, that's what I meant by partitioning, which I guess is more of a database term.

chesserik7• 9 years ago

Hey all. Thanks for noticing :P Obviously this is embarrassing and I'm sorry about it. As a non-developer I can't really explain how or why this happened, but I can say that we do our best and are sorry when that falls short.

- Erik, CEO, Chess.com

Aloha• 9 years ago

2 billion is a very large number that was probably not envisioned as reachable in the near future - as a programmer I'd argue this is a pretty easy mistake to make, and that while (slightly) embarrassing, its a good learning moment.

It's also really awesome that you're here, and that you guys were so honest about the nature of the bug - this is really something that should be encouraged.

chesserik7• 9 years ago

Maybe we should start a blog about all of the interesting bugs and challenges we encounter. It certainly is white-knuckle pretty often when running at scale. The number of devices, connections, features... I'm aging prematurely :P

nandemo3• 9 years ago

2 billion is a very large number that was probably not envisioned as reachable in the near future

I disagree. Simple napkin calculation: 100 million players playing 40 games each per year (about 1 per week) over 5 years = 10 billion unique games.

As others pointed out it was likely not a miscalculation, just a lack of calculation. The bug occurred only in the client and the decision to use a smaller data type was likely not a conscious one.

In any case, I wouldn't hold it against an individual programmer. But arguably this sort of bug indicates your development process has flaws (not enough testing, code reviews, etc).

relyio• 9 years ago

It's an honest mistake, things like this happen. Not the end of the world, especially for an otherwise great product like chess.com.

Go easy on your eng team ;)

chesserik7• 9 years ago

Thanks. I'm a pretty understanding "boss", especially on the heals of reaching the 2 billion games milestone :D Our team is awesome and we love what we do. Unfortunately we're still a bunch of humans sitting at kitchen counters and on couches around the world, so things do sometimes fall in between the cushions...

windowshopping• 9 years ago

We can help you understand how and why!

Computers set limits internally on how big numbers can be when they're keeping track of stuff.

Your developers had given each game a number to identify it. So your first game was #1, the 40th game was #40, and so on.

The limit for how big the number could be was a bit over 2 billion, and your players have just now played a bit over 2 billion games, and so that id number suddenly exceeded the computer's internal limit. Specifically, the limit was 2147483648, so basically it crashed on game #2147483649, which is the next id after the last acceptable one (notice the last digit is 1 higher.)

I'm simplifying slightly but that's the idea. It'll be fixable by essentially using a different format for the id number so that the limit is higher, much like telling the computer "use a higher limit for this particular number, it's special."

chesserik7• 9 years ago

Yes - I understand HOW it happened, just not sure WHY. Meaning, I'm not sure what the developer was thinking, and at this point, I'm not going to track down exactly who it was and point fingers. I think everyone has learned enough through this highly interesting bug. It certainly was interesting to see the slack room exploding with theories and debugging. A new iOS client has been submitted to Apple (hurry plz!!!), and a server fix is also in QA now. Fun problems to have......

scandox• 9 years ago

It's a beautifully simple problem and one born of success.

chesserik7• 9 years ago

Indeed. I'm not sure that anyone here at Chess.com at the beginning thought we would hit a billion games played in our lifetime. But I guess after 10 years....

yellowapple• 9 years ago

Think of a mechanical odometer, and how it only has a certain number of digits. Eventually you'll hit 999,999 miles, and on the next mile, everything will roll over to 000,000.

Same deal here. 32-bit numbers are stored as 32 switches, starting from

    0000 0000 0000 0000 0000 0000 0000 0000

which is 0, to

    1111 1111 1111 1111 1111 1111 1111 1111

which is 4,294,967,295. Since the 32-bit iOS version of Chess.com apparently uses 32-bit numbers to store each game's unique ID, that means you can have 4,294,967,295 games.

So what happens on game 4,294,967,296? Just like the odometer, everything rolls back to 0, and things start breaking because the program gets confused.

Pretty common problem, really. The fix would be to use a 64-bit number, which doubles the number of binary digits.

pram• 9 years ago

I recently experienced a nasty bug with BLOB in MySQL. The software vendor was storing a giant json which contained the entire config in a single cell. It ran fine for months, and then when it was restarted it totally broke. Reason was: the json had been truncated the entire time in the database, so it was gone forever. It was only working because it used the config stored in memory on the local system. Nasty!

fpgaminer2• 9 years ago

For those, like my, that didn't recall off the top of their heads: BLOB in MySQL can usually only hold ~64KB.

EDIT: Though I am curious why MySQL doesn't throw an error when you try to store more than 64KB in BLOB?

frandroid• 9 years ago

The BLOB size limit is a nuisance, the silent truncation is the bug...

rsynnott2• 9 years ago

It'll give a warning. Many/most MySQL drivers won't interpret it as an error, though.

wnoise2• 9 years ago

Huh, I thought BLOB was a backronym for Binary Large OBject, not Binary Medium Object.

DCoder• 9 years ago

MySQL's silent data truncation is such a nuisance. It's off by default in 5.7, and can be disabled in earlier versions by adding STRICT_ALL_TABLES/STRICT_TRANS_TABLES to sql_mode [1].

I inherited a system where, among other things, the entire response body from a payment gateway callback is saved into a text field using utf8 character set, despite the fact that most of the supported payment gateways send data in iso-8859-x (and indicate the used charset inside the body itself, how's that for a chicken-and-egg problem). Of course when the data gets truncated due to not actually being utf8, nobody notices. Fun times.

[1] https://dev.mysql.com/doc/refman/5.7/en/sql-mode.html#sql-mo...

eric_h2• 9 years ago

MySQL's silent data truncation is such a nuisance.

Yes, yes it is - it burned me so badly (catastrophic, unrecoverable production data loss) in the early days of my career (~15 years ago as a junior level dev in a senior level role) that it has forever colored my opinion of MySQL - I will really never trust it again.

Long live PostgreSQL!

Piskvorrr5• 9 years ago

Early web had these issues as well: server sends response with Content-Type: text/html; charset=win-1251 (or no charset), but body contains meta charset=utf-8. MSIE worked around that in IE4 by comparing the letter frequencies to a hardcoded table and guessing the charset from there. It sort of worked, 80/20.

tyingq2• 9 years ago

Mysql's unix_timestamp() breaks on years past mid 2038...so plan for that one too.

estebank• 9 years ago

OpenBSD[1]:

time_t is now 64 bits on all platforms.

Linux[2]:

The vast majority of 64-bit hosts use 64-bit time_t. This includes GNU/Linux, the BSDs, HP-UX, Microsoft Windows, Solaris, and probably others. There are one or two holdouts (Tru64 comes to mind) but they're a clear minority.

This is little help for older, already deployed systems, of course.

[1] https://www.openbsd.org/55.html

[2] https://mm.icann.org/pipermail/tz/2004-June/012483.html

callumjones• 9 years ago

For f sake how are we supposed to Anderstand that. I suppose your French fry maker is broken ?

Didn't expect Chess.com and YouTube to have a crossover of users? Surprised there isn't active moderation on a site this size.

Zanta• 9 years ago

In my experience the chat on chess.com harbors a similar demographic to that of most video games. You'd think that chess would attract a more mature player base, but nope.

marcelluspye• 9 years ago

The only time I've observed people in real life acting like people in video games is at a chess tournament. Constantly trash-talking until they lose, then accusations of cheating. You certainly don't see this type at all (or most) chess events; I think the lack of entry free drew them out into the open.

Bakary2• 9 years ago

Chess played on a computer fits the literal definition of a video game, and one that has near infinite replay value.

key8700• 9 years ago

eBay (almost) had this problem and I cannot find any articles about it online. They were rapidly approaching 2^31-1 auctions. So they switched to a larger integer, the switchover went badly, and they were mostly down for 4 days, if my memory serves. This would be like 10+ years ago I think.

mtkd• 9 years ago

These are always the best problems to have

_pmf_• 9 years ago

That's the most successful reason for failure.

russellbeattie• 9 years ago

This problem is more related to a programming underestimation than the actual limitations of a 32bit CPU (which can happily process numbers or IDs that arbitrarily big if you have the memory for it and program it correctly).

That said, this is definitely indicative of what's going to happen in just 20 years, 6 months and 20 days from now. I mean, we're still cranking out 32bit CPUs in the billions, running more and more devices, and devs still aren't thinking beyond a few years out. I know of code that I wrote 12 years ago still happily cranking away in production, and there may be some I wrote even longer than that out there... and I guarantee I hadn't given two thoughts about the year 2038 problem back then, and I doubt many devs are giving it much thought today.

It's truly going to be chaos.

protomyth• 9 years ago

The sad part is people are going to look at the lack of a year 2000 event and assume 2038 is going to be a "dud", when they fail to see all the damn work that went into making sure Y2K was a dud including a significant portion of IT hours and probably a lot of extra support laid in.

I expect 2038 to be a rare hell because of the nature of the devices. Y2K was an IT problem, but 2038 will be an embedded system problem and that's going to be a much more painful thing to audit. Moving from the server room to inside equipment and walls is going to be fun.

isolli• 9 years ago

Why 2038?

fsiefken• 9 years ago

Will the Lichess app and platform have this issue? And if not, why not?

shmageggy• 9 years ago

Looks like lichess is using strings for IDs so they will not have this issue

https://github.com/ornicar/lila/blob/master/app/controllers/...

chesserik7• 9 years ago

Fun to read some of other stories where this bit them too (PacMan, WoW, and eBay)! Anyway, new app has been approved by Apple and should be rolling out soooooooooon....

Thanks for all the comments! Always lots to learn from.

spullara3• 9 years ago

The other one to watch out for is the 53-bit javascript integer limit. That caused the twitpocalypse when Twitter tweet IDs hit it. They had to switch to strings in the JSON representation.

gilgoomesh2• 9 years ago

The 2009 Twitpocalypse concerned overflow of 31-bit precision. Twitter has not yet hit 53-bits for raw number of Tweets, in fact, they passed 32-bits in 2014 and might not have reached 33-bits, yet.

Moving to strings for Javascript was really just safety planning for the future since:

Given the current allocation rate, they'll probably never overflow Javascript's precision nor get anywhere near the 64-bit integer space.

https://twittercommunity.com/t/discussion-for-moving-to-64-b...

spullara3• 9 years ago

That discussion is about user IDs, not tweet IDs.

Tweet ID from today: 875423039323688960

Number of bits of precision necessary to represent it exactly: 60

Overflowed 53-bit precision long long ago. You can read about it here: https://dev.twitter.com/overview/api/twitter-ids-json-and-sn...

inieves• 9 years ago

The title is probably wrong, off by one.

You probably mean 2^31 -1.

Piskvorrr5• 9 years ago

Help me, Off-By-One Kenobi! You're one of my two last hopes!

nicky0• 9 years ago

an unforeseen bug that was nearly impossible to anticipate

Hmmm... :)

yoz-y2• 9 years ago

What would be the best way to test for this kind of issues in advance. Testing at theoretical limits at all endpoints?

damagednoob• 9 years ago

I'm not sure this falls under testing. If you start with an empty database each time you start the test you may never hit this issue.

I think this is more of a capacity planning issue.

yoz-y2• 9 years ago

My concern is that even if one plans for a sufficient capacity, there still needs to be testing done to verify that the code actually works if the capacity is nearing the theoretical limit. In this example the database id was transformed into a 32 bit integer somewhere in the application code.

Usually when I hit some sort of unexpected bug in production I try to think about what type of testing will prevent similar problems in the future.

vitomd• 9 years ago

A lot of comments but no one said the great time that we are living for chess. So many games online, ready to be analysed and learn from them. After deep blue people thought that it was the end of chess, but it´s only getting better. Computers helping players to improve.

Chess.com is a great site, also lichess.org and chessable.com if you like chess you should check them.

cwfrank• 9 years ago

Issues like this are not uncommon on Chess.com. I've been playing there since 2008 or 2009. If you read recent comments about issues as they pertain to the recent "v3" release ... as much is to be expected.

Chess.com stopped working on 32bit iPads because 2^31 games have been played

Related Stories