Apple Developer Center is Fully Restored

https://developer.apple.com/support/system-status/#

by nimeshneema • 13 years ago

46 46 13 years ago

developer.apple.com

k-mcgrady• 13 years ago

It took over 3 weeks to restore everything. I hope Apple does some sort of post-mortem (at least something basic). When a security breach takes you down for this long it was either a very serious breach or your security and infrastructure was shit and your customers deserve to know how you've remedied that.

X-Cubed• 13 years ago

I was wondering why I was unable to access any of the services that were supposedly back online. Now that the Member Center is back, it turns out there were new Terms & Conditions I needed to accept first... argh.

seivan• 13 years ago

My account is still subject to expiring even though I did a renewal before things went down - should have eleven months at least.

Anyone else got this issue?

fredley2• 13 years ago

So the whole place went down a few days before we were ready to launch. We need to upload an iPhone app to iTunes Connect. We have been prevented from doing this because we needed to accept some new T&Cs on the member centre. Despite iTunes Connect being up, we've been stuck. Apple support were unhelpful - we've just had to wait it out. Now finally I've been able to get on the member centre and accept the T&Cs, but there's clearly some delay in iTunes Connect realising this. I still can't upload my app.

Many more profanities are being hurled at Apple this morning.

harrytuttle4• 13 years ago

Sorry but when you base your businesses on walled gardens you are forever at the mercy of others.

There is nothing to complain about here; you've just got to suck it up.

fredley2• 13 years ago

The business is not based around mobile, but if you're going to have any kind of mobile apps - however incidental to your core product - you have to put up with it.

AlexanderDhoore3• 13 years ago

I wonder what Apple's server infrastructure looks like? They aren't programming it all in WebObjects [1] are they? Do they use linux and/or freeBSD? Of all the things Apple does, their "cloud" seems the weakest product.

Apple is the only big software company that doesn't develop server-side technologies. If I was the boss of Apple I'd standardize on FreeBSD and Golang today! But, I'm not... obviously :)

[1] http://en.wikipedia.org/wiki/WebObjects

nwh4• 13 years ago

I've brought this up before, but it's a mix of everything, and all pretty bad.

If you've ever poked around with the way that Apple's website works, you can see that the entire place is a huge mess. There's old servers running ancient (pre-2004) perl scripts alongside the brand new iCloud gear. I can't imagine how the authentication for AppleID is working as login details still work on the ancient pages (think pinstripes and glassy buttons). Depending what URL you hit, the webserver is using php3, php4, perl, python or maybe WebObjects (java).
At one point I wrote a scraper that was targeting one of their product pages, and kept getting random, unexplainable results. It turned out that one of their product areas was behind a round-robin load balancer, with three completely different apache versions on each server. The page was dying on one but not the other two. In the end I just had to repetitively scrape until I hit a good response.

https://news.ycombinator.com/item?id=6078854

AlexanderDhoore3• 13 years ago

Wow, that's exactly what I was afraid of... You can tell it's absolutely not their priority. It's more of an afterthought. "Oh, yeah, btw, we need some servers as well, but first do all the other stuff."

Terretta• 13 years ago

Apple is the only big software company that doesn't develop server-side technologies.

iCloud storage, iTunes Match, iMessage, to name just a few. If you stop to think how these all just work across devices and desktops, and at what volumes, you'll realize they've got some server chops.

AlexanderDhoore3• 13 years ago

I meant they don't create any server-side language or framework... They will have lots of infrastructure and code, no doubt.

lazerwalker• 13 years ago

If you've ever used iTunes Match, you'd know it's a stretch to say that it "just works".

nwh4• 13 years ago

They mostly run on Microsoft Azure and Amazon EC2.

They also don't really work. iMessage is unreliable at best, and iTunes match just makes home taping look preferable.

dave1010uk3• 13 years ago

Not not often one to praise Apple, but I'm really impressed that they spent the time they did to fix this. Apple are known for their strong brand and taking down the dev centre for so long must have been an awkward decision between the marketing side and the technical one.

I don't know the extent of the problem but taking down an online service for weeks is very uncommon. I guess (speculating a bit) that Apple could have applied a quick fix to the problem within a day to save their (short term, at least) brand appearance. I think "just getting it secure enough" is the most many people would do if an important service was down. It appears that Apple took the time required to deploy a proper fix, prioritising security over shot term wins.

It would be interesting to see what someone like Microsoft or Amazon would have done in this situation, or what Apple would have done if it was all of iTunes instead of the dev centre.

dave1010uk3• 13 years ago

Does anyone know of a centralised database of security breaches? It would be great if there were a standard (like CVEs), which included how well the breach was dealt with (eg informing public, mitigation, prevention, changes made). If there isn't one and there's interest here then maybe I'll start one.

thezilch• 13 years ago

You're impressed they spent time keeping their cash cow alive? No offense to the developers, but there has been very little to no communication. I'm magically waiting for the guy with the best F5 time on the dashboard to get my news? Again, no offense to the developers, but there's no indication that this was not a "quick fix" by industry standards (of security) and not just "secure enough."

As usual, a dashboard of green lights does not identify uptime, as can be established by the issues people are still having with the platform.

deminature• 13 years ago

No offense to the developers, but there has been very little to no communication.

There's been weekly or slightly shorter than weekly updates and a 24-hour status page. Also, most key portal functionality was available not that long after the breach.

It's a little unfair to say "very little to no communication".

fauigerzigerk• 13 years ago

You are very generous with your praise.

harrytuttle4• 13 years ago

They did nothing good here. They fucked up and took forever to fix it. Please stop apologising for them.

dave1010uk3• 13 years ago

Oh I'm not apologising for them, they still messed up (and I still actively discourage people from buying Apple products). I don't use any Apple products / services, so their downtime didn't bother me. Their lack of communication is bad but I'd prefer poor communication over poor security.

We know they didn't push a quick hacky fix live. I think it's quite likely (due to the resources they have) that they deployed a proper fix, not sacrificing security for a faster response. I could be wrong.

pknerd• 13 years ago

It is not for me. When I try to Join SAfari developer Program it is redirected to this URL:

https://developer.apple.com/iphone/urlRedirect.action?mode=e...

nwh4• 13 years ago

From the action URL, this seems to be the page you want — https://developer.apple.com/ios/enroll/individualEnrollmentI...

mickeyckm• 13 years ago

Hmmmm, after I accepted the new terms & conditions, the iTunesconnect still ask me to accept it in Member Center. I didn't expect to have some sort of delay between the changes in Member Center and iTunesconnect. Hope it takes effect soon. Sigh.

msie2• 13 years ago

Argh, it's silly that it takes some time for a membership renewal to take effect.

msie2• 13 years ago

Why isn't this valid criticism? Buy a digital membership anywhere else and the effect is immediate. Buy something on iTunes and you get the product immediately.

jmomo• 13 years ago

If Apple had a security breach, and it seems like a massive one given the circumstances, are they not required by California law to report it to their customers?

pilif• 13 years ago

They did report that there was a breach, that some personal data about developers (but no credit card info or passwords) was stolen and that they are restoring the developer center.

That's more than enough reporting. If you want a full post-mortem, that's likely not something you'd get from a public company as big as Apple

robryan• 13 years ago

AWS always gives full and detailed postmortems. I wouldn't say it is a trait of big companies specially not to disclose to customers, but it is something I would expect Apple to probably do.

tazjin• 13 years ago

You wouldn't get a full post-mortem from a company like Apple, because they are wilfully intransparent. My employer (Spotify) for example posts somewhat detailed post-mortems[1] after big outages - doing this always comes with a spike in job applications so it's positive in several ways.

Another comment mentioned that AWS post-mortems are also detailed and public, they don't really have another choice because their customers have their infrastructure running on AWS - so they want to know what happened and not be left in the dark.

[1] http://labs.spotify.com/2013/06/04/incident-management-at-sp...

marshray2• 13 years ago

Let's pour a cider on the ground for all the fallen evenings and weekends that were surely sacrificed by our industry comrades to accomplish this restoration.

harrytuttle4• 13 years ago

For fucks sake. If they'd done a half decent job to start with, they wouldn't be there.

They fucked up, now they're paying for it.

marshray2• 13 years ago

Surely you don't believe that everyone who busted their ass to fix the problem was also responsible for causing it?

jng• 13 years ago

Great Saturday morning news, I've finally able to run my new iOS app on my iPhone! I had to accept new T&C before getting certificates/provisioning-profiles, and this has been impossible for weeks.

marknutter3• 13 years ago

The profiles center has been a online for quite a while now.

myko2• 13 years ago

I had to accept new T&C before getting certificates/provisioning-profiles, and this has been impossible for weeks.

If you can't accept the new T&C you can't get into the profile center.

eddieroger• 13 years ago

But the agreements portion wasn't. My company's enterprise license was supposed to run out a week ago. Luckily they extended it, but the hub of everything - the agreements server - was down.

kyriakos• 13 years ago

..if microsoft's or google's services were down for this long hell would freeze...

just saying that apple seems to be getting treated better for no reason

marknutter3• 13 years ago

You mean, if Microsoft's or Google's developer services were down for this long. I honestly don't think people would raise any more stink than they did with Apple.

corresation• 13 years ago

On the flip side, Apple took a security issue very, very seriously, and for that they should be commended. I mean, obviously better if it were found beforehand, but clearly someone outside of the people in charge of these web properties had a WTF moment and life became unpleasant for some people for a while.

Most firms would have duct-taped the specific section exploited.

myko2• 13 years ago

On the flip side, Apple took a security issue very, very seriously, and for that they should be commended.

Though they were warned before the breach occurred, right? They didn't take it seriously until well after they knew they needed to fix it.

Apple Developer Center is Fully Restored

Related Stories