How Hackers Stole 200,000 Citi Accounts Just By Changing Numbers In The URL

http://consumerist.com/2011/06/how-hackers-stole-200000-citi-accounts-by-exploiting-basic-browser-vulnerability.html

by mmavnn • 15 years ago

360 156 15 years ago

How can a bank this large have such poorly designed security? It's ridiculous. Hopefully, all these latest hacks get everyone else to treat security more seriously. There could be a lot of other banks that do the same thing as Citigroup. So if one gets hacked, at least the others will remember to review their security policies, so it doesn't happen to them, too.

mrcharles• 15 years ago

How is this even possible? In my very first website I built from scratch using PHP, I paid attention to the possibility of this. I can't say for certain that I fully protected against it, but I tried. That little trick would not have worked.

How is it that a bank, of all places, pays money for a web infrastructure, and manages to employ people who don't even think about the most basic of attacks? I've been changing info in URLs since I started using the internet.

rmc2• 15 years ago

Diaspora had this problem when they first released their software, and it was written in Rails, a modern framework http://www.kalzumeus.com/2010/09/22/security-lessons-learned...

spdy• 15 years ago

You cant really compare this with a bank like citygroup. Security is everything the should care about in the first place. They have to protect your money and nobody really cares about fancy features when they just want to wire some money, everything should just be rock solid.

I would love to get more information about this breach it sounds to simple to be true.

StavrosK• 15 years ago

The framework has nothing to do with not authorizing requests properly. Diaspora had many problems when they first released because apparently they were all inexperienced programmers, like the guy who wrote that in Citi.

Jach• 15 years ago

How indeed.. How can someone be able to architect the system of accounts, connection info, etc. for a bank but not be aware of such things? And it's not even being lazier either. Sigh... To all you 13 year old kids on the road of learning web dev, this is what you're competing against!

StrawberryFrog• 15 years ago

A security guy weighs in on it here:

http://idunno.org/archive/2011/06/14/citibank-hacked-ndash-d...

"This was not sophisticated or ingenious, as reported, this was boringly simple. ... OWASP has had Insecure Direct Object references on it’s Top 10 list for years. It’s in the SDL Threat Modeling tool. Any security firm worth its salt checks for this"

Yes, there's a good description of this kind of trivial "hack" in the Open Web Application Security Project Top 10: https://www.owasp.org/index.php/Top_10_2010-A4

wccrawford• 15 years ago

Yeah. When that first report said 'experts' said how hard it would be to have predicted and prevented this, I choked. Amazingly lame.

peterwwillis3• 15 years ago

Citi has an internal web application pen-testing group. My guess is they were only ever attacking the outward facing apps and not the ones after successful authentication. Even if so, they may have hundreds of apps to test which constantly change, and sometimes attackers just get lucky and find a hole as soon as it pops up.

sambeau• 15 years ago

From the article:

"this is a dead simple and common hack and Citi should have seen it and prevented against it. Seriously, this is kindergarten level stuff. Really, really stupid."

aw3c2• 15 years ago

"Gawker media blog spam".

Visit http://www.nytimes.com/2011/06/14/technology/14security.html... instead

blatherard• 15 years ago

Consumerist isn't a Gawker property anymore. They are now owned by Consumers Union, who also publishes Consumer Reports.

sili• 15 years ago

I was actually disappointed by the NYT article. They interview security experts who call the attack "ingenious", "hard to prepare for" and performed by exploiting a vulnerability in a browser. This understates how incompetent the bank's website design is.

mmavnn• 15 years ago

That was why I linked to this article rather than the NYT. The article linked to, while shorter, contains all of the relevant information and less completely random cluelessness.

narad• 15 years ago

That's worse than SQL injection. Didn't they build ACL?

romaniv• 15 years ago

ACL is probably part of the problem here. Most ACLs are very inflexible and are "opt in". They probably had ACL to block unregistered user from visiting the page, but it didn't deal with individual accounts.

thibaut_barrere• 15 years ago

You'll surely enjoy the quote from the linked article:

=======

The method is seemingly simple, but the fact that the thieves knew to focus on this particular vulnerability marks the Citigroup attack as especially ingenious, security experts said.

=======

Sorry but no, this isn't ingenious - it's really the basics!!!

john2x• 15 years ago

Yeah, the NYTimes article really made it look like the hackers went into a lot of trouble to breach the bank's security.

webjprgm• 15 years ago

Of course they did, if they made it look simple then everyone would freak out (or they should, and have mass exodus away from Citi). People are used to hackers finding occasional, difficult exploits.

bron• 15 years ago

Of course, the hackers were suffering from acute Alzheimer.

tnorthcutt• 15 years ago

I really hope that the reporter who wrote the NYT article either misunderstood the facts, or talked to an ignorant "expert".

fmavituna• 15 years ago

This is like the first thing you learn in web app security (defender or attacker) and you don't even need to write script a tool such as http://code.google.com/p/fm-fsf/ will scrape the data quickly.

Even though it's insanely easy to spot and exploit it's also easy to miss it while coding. But any decent pen-tester will find it. Regardless, unacceptable for a finance company.

planb• 15 years ago

No, I don't think easy to miss because it shouldn't be possible in the first place. At least for a banking software, the web application layer is not the right place to check for this kind of authentication. This makes me wonder how their code must look like...

dave1010uk• 15 years ago

All you need is curl:

  curl http://example.com/user/[1-100]

leftnode• 15 years ago

What's worse: every customers ID in the database was stored in the URL or that there was no ACL to test against? If a user is logged in, you have their account ID stored in a session. If they navigate to a page that their account ID can't see (like another person's account), then kick them out. Astoundingly simple.

quattrofan• 15 years ago

Seriously how do people stay in their jobs allowing crap like this to happen? The CIO or CTO at Citi should get the boot. Until companies like this and Sony start making examples of people, this kind of sloppiness that gives our industry a bad name will continue.

groby_b• 15 years ago

Except the examples that will be made will not involve the CIO/CTO. And even if it did, the guy almost certainly a golden parachute - what does he care?

As long as it's cheaper to clean up after the debacle than prevent it in the first place, that's what people will opt for.

rubinelli• 15 years ago

They probably didn't develop it in-house, so it's just a matter of blaming and changing providers.

gnosis3• 15 years ago

"And even if it did, the guy almost certainly a golden parachute - what does he care?"

If he really was fired and the company made a public stink about how much he screwed up, he may well care.

Many people at that level care about their prestige, and care about being shamed in front of their peers, even when they don't need to worry financially.

sdizdar• 15 years ago

In majority of corporations responsibility and accountability ends somewhere between VP and CEO.

mgkimsal5• 15 years ago

In 2000(?) I employed this same 'hack' against Ameritech's online bill viewer, but couldn't get anyone's attention. I called several people at Ameritech, but couldn't get through to anyone who understood anything I was saying.

I tried to get ahold of newsmedia, but realized afterwards that the links I was sending did have a session timeout associated, so by the time a reporter clicked a link, they got nothing.

Finally, I managed to get in touch with someone at 'fuckameritech.net' (IIRC) - a consumer watchdog (I hesitate to say 'group' - I think it was just one guy) who said "I'll take care of it". He made some contacts - I think got it to a reporter in Chicago, and that afternoon Ameritech's online bill view and pay was taken down (a wednesday IIRC) and it wasn't brought up again until Monday.

The 'fix' was not much - they were now hashing the account number in some massively long (128 char?) ID instead of just your account number. But it was all still visible in the URL, which was the bigger problem to start with, because it encouraged 'hackers' like me to change my account number by one digit.

I suspect others had noticed this before, tried to contact citi, and couldn't get in touch with anyone who understood what the caller was saying.

Companies need separate 'web vulnerability' hotlines to call/contact to report issues like this - perhaps just hidden in the 'view source' - if you're good enough to find the info, you know what you're doing enough to report a problem. Too low a bar?

FooBarWidget• 15 years ago

I imagine that publicizing a web vulnerability hotline would result in more trouble that it would solve. Normal people really don't understand computers. If you somehow give off the message that your system is not perfectly secure or bug-free they would get scared and run off to competitors who are just as bug-ridden but at least appear to be more secure.

mgkimsal5• 15 years ago

While I sort of see, marketing/wording of this could spin this as a positive (assuming it was even something that was publicized - perhaps hidden in the markup is good enough?).

We've popularized crime reporting as a social good - mcgruff the crime dog, etc. When will we start taking online safety and security with the same level of seriousness?

As much as I don't like the idea, being able to report issues to a state or federal agency might be a way to go.

jrwoodruff• 15 years ago

Normal phone techs should know how to deal with such calls (even if they don't understand the exact problem) and have a line/department to forward the call too. That provides filtering for legit claims, and avoids advertising it and scaring people. Simple.

cosgroveb• 15 years ago

Eh I dunno. Google does this with its Vulnerability Reward Program [1] and people seem to be fine with sharing almost all of their private data with Google from all their e-mails to their credit card numbers (Google checkout), etc., etc.

Also Facebook has a form for reporting vulns [2] and people are still happy to share their personal info there. I'm sure there are other companies that have "hotlines" but these are just a few I can think of.

I don't think having an avenue for responsible security bug disclosure gives anyone the impression that their data is unsafe.

[1] http://googleonlinesecurity.blogspot.com/2010/11/rewarding-w...

[2] https://www.facebook.com/help/contact.php?show_form=white_ha...

jasongullickson2• 15 years ago

I don't disagree with you, but this reminds me of the anti-seatbelt arguments from automakers before the law required them.

peterwwillis3• 15 years ago

I agree that companies need an easy way for people to report potential problems or risks. One company I was with had no idea it was a botnet C&C until months later somebody looked up our WHOIS and sent an abuse e-mail. Some non-hacking-related forums had known for a while.

Shenglong• 15 years ago

It's funny that this wasn't discovered sooner. I suppose everyone figured a security flaw like this would never exist, and never bothered to try. Irony...

wladimir• 15 years ago

Most likely this was discovered sooner. But people kept the vulnerability to themselves and possibly even profited from it.

These kind of vulnerabilities in high-profile sites can go unreported for ages, even up to the point that they are common knowledge in certain groups.

lucraft• 15 years ago

I'd argue that this wasn't even a hack. This is publicly visible information. No security measures were circumvented.

noduerme• 15 years ago

So...the first login had to have valid credentials. Someone needed a citibank account to start the scraper-bot. Wonder if the fbi's talked with that guy yet.

count• 15 years ago

I'd bet they used one of the hundreds of thousands of stolen accounts from other services to do this. You know somebody is sharing usernames/passwords with their SOE and Citi accounts.

adolph• 15 years ago

I think the key phrase would be "Someone needed access to a citibank account..." If some Citibank account(s) was compromised in some way, such as phished, then a third party would have the necessary access. It would be bad enough to be a victim of phishing--imagine being a pawn in a large scale attack like that. It would suck.

smackfu• 15 years ago

I wonder where they got the account numbers from. Citi doesn't show them in the web interface, just last 4 digits.

patio112• 15 years ago

I'm kind of torn on this. On the one hand, yeah, it is a trivial flaw.

On the other hand, so is waving a gun at a teller. That attack has been around for decades and still works a few dozen times a year, because the cost/benefit analysis says that after hardening the banks a little it is easier to just lose a few tens of thousands of dollars every once in a while than it is to give them the Secret Service's attention to physical security.

That is hardly the only systemic vulnerability in the banking system. For example, let's suppose I want to compromise your account number and credentials sufficient to take you for every penny you possess. You know what I need? A check of yours. Any will do. Everything I need to create a demand draft against your account is on every check you have ever written. Every employee of every business you have ever paid by check got the keys to your financial kingdom.

You may not be aware of it, but since those credentials are assumed compromised, the security is in a) catching me when I use the demand draft to suspiciously drain your account and b) failing that, making you whole out of the bank's pocket. The numbers have been crunched: it is vastly, vastly more efficient to treat fraud as a cost of doing business than it is to tighten the screws 100%

The attack surface on software the size and complexity of a bank's is like the Death Star, except any single rivet being out of place will eventually result in this headline.

(Step #1 in tightening the screws would be turn off public facing websites, because inexpert users plus compromised machines means that no banking website will ever be secure, even without coding errors. This will never happen, because the provable cost savings of moving customers to online banking roflstomp over the marginal fraud risk.)

st3fan• 15 years ago

The attack surface on software the size and complexity of a bank's is like the Death Star, except any single rivet being out of place will eventually result in this headline.

I agree that banking software is probably complicated. But really, this bug is a beginners mistake and it shoul dhave never happened.

It is something that the original developers should have known about and also something that the company auditing this code should have seen.

There is really no excuse for this.

bigohms2• 15 years ago

This is like the first thing any new hax0r tries. Imagine the joy when it actually worked...

dclowd9901• 15 years ago

There's also something to be said for having more deterrence than the next guy. Which is better business? Cutting ridiculous corners to save on developing decent security, and eventually getting egg on your face, to the chagrin of your customers, or simply keeping idiotic measures like this out of your software so it's one of your opponents' faces that are hit with the egg?

alex_c• 15 years ago

Nonetheless, banks still lock their branches at night. That's about as basic as security gets.

ctdonath4• 15 years ago

This isn't akin to waving a gun at a teller. This is akin to handing the teller a huge stack of withdraw forms with random account numbers on them, and the teller dutifully checking each one, ignoring the ones with invalid numbers (most of them), and handing over the account contents for all the valid ones. This doesn't work, because the teller acts as security: if something is amiss in the implied security checks, to wit there is no reason one person should be submitting requests regarding lots of account numbers esp. when most of them are invalid; she'll have alerted management & security within seconds of seeing the bizarre request.

Your subsequent analogy/justification/complaint is only valid if ONE doctored URL were used. My bank, Chase, does in fact implement security against such a "lots of random account numbers" attack: not only must the account match, but the MAC/IP address, browser/cookie, and other under-the-hood identifiers must line up; any mismatch between account number and access tools initiates emailing or texting a verification code to a known address/phone, which then must be submitted to close the loop of verification and, only then, allow access. Not running some kind of "one account per access device" sanity check is insane.

It's not about one rivet being out of place - such vulnerabilities are understandable. It's about having an uncovered vent lead straight to the reactor core - that's stupid.

vnchr• 15 years ago

Like the Death Star...

jasongullickson2• 15 years ago

I think you're missing the point (or I'm inferring it incorrectly), that, put simply, there is a cost associated with implementing the security measure (both to build it and the inconvenience to customers) which in some cases outweighs the potential cost of the exploit.

stef252• 15 years ago

> Not running some kind of "one account per access device" sanity check is insane

So I would need different accounts for my personal computer, my laptop, office computer and the computers of my parents? I regularly work on all of those.

speleding• 15 years ago

> The attack surface on software the size and complexity of a bank's is like the Death Star

Really? I'm sure there is more to it then I imagine but I've used the websites of 3 different banks and their functionality is quite limited, it looks like something you could do in a Rails app in a few weeks.

None of the real payment processing is done by the web app as far as I know, it is just a thin layer on top of a database with the transaction history and can store payment orders for batch processing later on. None of the real payment processing systems would face the internet I imagine, so I think the attack surface would be quite small.

awj• 15 years ago

Running cost/benefit on this is great, except that the numbers still don't add up.

Instead of a user id that is obviously some row in a database, use a UUID and do your db lookups against that. Worst case you have to put in some kind of local UUID -> user id lookup table because you can't manage to change the user record database directly. Now, barring a broken UUID generation scheme, it's virtually impossible to crawl for user accounts. Plus you have a scheme for providing the same guarantee on any other identifiers.

Obviously that doesn't prevent the larger problem of "guessing your user id and munging a url lets me into your account without a login", but it at least makes the guessing part worlds harder for relatively little work.

rmc2• 15 years ago

On the one hand, yeah, it is a trivial flaw. On the other hand, so is waving a gun at a teller.

Not really. To make it so that waving a gun at a teller doesn't work you need to do lots and lots of complicated hard things.

However it's as hard to ensure that person with account A doesn't access account B.

snorkel• 15 years ago

I know exactly how this occurs because I recently met a "Senior Web Developer" at an established business who was basically their acting architect because he was the their first coder and therefor his non-technical bosses regarded him as some kind of genius because he knows how to unjam the office printer. He didn't know a lick of Unix, didn't understand load balancing, and had very weak SQL skills. He was your typical framework junky who couldn't imagine writing even the simplest web app without a framework to do all the heavy lifting. All he wanted was for me to recommend an even simpler web framework so he wouldn't have to write any SQL at all. No doubt some day his code will be generating headlines like this one, and he will no doubt blame whatever framework he used and his bosses will simply mandate that they switch to a more secure framework pronto, and they'll promote this boob as their Senior Architect to lead the project.

silverlake• 15 years ago

I've met hundreds of these boobs in IT shops around the world. There's a huge skills gap between IT and software devs. IT guys are not given the time nor resources to learn and implement all this stuff. Also, they often don't have the skills or curiosity to figure it out themselves. So they really do need a framework that takes care of it all easily.

lawnchair_larry• 15 years ago

That actually sounds like the right way to do it to me. He shouldn't be writing frameworks. Frameworks are reviewed by a lot of people, and there is a far greater chance of them being bug free than something home-rolled.

lifeisstillgood2• 15 years ago

The big question is why the structure of the IT department lent itself to doing something so stupid

You can fire the CIO, you can replace the offshore developers with onshore, or vice versa, but experience says it won't matter.

I looked in amazement at googletesting's dependancy graph test suites yesterday, and realised that the playing field is not flat at all.

Reading and writing code is the literacy of the 21st C.

And the end most big companies are like newspapers owned and managed by illiterates.

It does not matter how you rearrange the strucutre or the hierarchy, when the chips are down decisions will be made on what the illiterate management understand is the best way to work. As such it is infinitely unlikely that the decision then will be set up to support what a literate person would decide.

Until a generation of coders grows up, or all illiterate companies go bankrupt, this will merely be one of a myriad of pathologies exhibited by large companies run by the illiterate.

yuvadam• 15 years ago

A naive question here:

Suppose I accidentally stumble upon a gaping security hole in my bank's online service (or any other online service for that matter).

Am I legally obliged to notify them of that security bug? Can I offer the bank my assistance, for hire, in solving the bug without it constituting blackmail? (i.e. I'd be happy to help you solve this at a $300/hr rate)

bonzoesc• 15 years ago

I've always got the impression that it's more convenient and less risky to not tell anyone; when the flaw is eventually exploited, you don't want to be the one on record as having known about it.

mike-cardwell• 15 years ago

Or report it anonymously using Tor or a remailer. If you do it anonymously, you also get the benefit of being able to safely threaten the bank that you will release the information publicly if they haven't fixed it within x days.

ctdonath4• 15 years ago

Legally obligated? no. It's not your system, it's not your problem in that sense.

Sure you can offer to fix it, but since you don't know squat about the system (save for a small flaw at the surface) and they have teams of developers who do, they won't be interested in paying you to fix it. Offering to explain the bug for a fee won't be blackmail unless you threaten to reveal the bug to others if they don't pay up.

Be a decent chap. Send 'em a nice letter explaining the problem. It's your bank, remember, and they're humans like you; work with your service providers to improve the service. Assume you're not the only one who knows about the problem, that someone who also knows isn't as nice as you, and it's YOUR bank balance that is at risk.

clistctrl• 15 years ago

Of course then there is the saying "No good deed goes unpunished."

rapind2• 15 years ago

But it's a corporation, not humans like you. I'm sure they employ lots of decent people. I'm sure they employ lots of douchebags too. It's irrelevant. It's a corporation who's only motivation is to make money. It's not your neighbour.

If you like their service and feel like telling them then go for it. If you want to try to charge them, then go for it (no it doesn't make you evil to charge for a service). If they don't want to pay you, feel free to say nothing.

Now if it's a mom and pop shop down the street, then yes, please be a good neighbour and help them fix it (although you can still charge for it, but avoid douchebaggery like $300 / hour unless that's your usual rate).

holdenc• 15 years ago

I have a feeling there's more to it than a clear account number reference in the URL. It was probably a base64 encoded account number or a non-salted hash of the account number (ie. rainbow table-reversible) and the quality assurance analysts probably never questioned this.

Disclaimer: I worked in product development making banking software and simple URL hacking was always a standard test.

mbreese• 15 years ago

If this really was the "hack", you can be sure that Citi is has opened themselves up to a whole world of negligence lawsuits. This is the same as having a vault where any customer could walk in and just browse around the safe deposit boxes. Sure, it might be tough to be authenticated to get into the vault, but once you're there...

This is something that should cause the immediate dismissal of the CIO, but sadly, probably won't.

tannerc• 15 years ago

Is there anything Citi has done right over the past few years? I'm asking honestly and not with sarcasm, because I just haven't seen it.

raganwald2• 15 years ago

This is unacceptable, obviously. It's a massive failure of their security testing protocols. I'm not particularly surprised that an vulnerability like this would get written into the code, it's an easy mistake for an inexperienced developer to make. But I'm not going to pile on the developer. We laud "separation of concerns" in our architecture, and this pattern applies to the organization of software development teams.

I don't expect every developer to be aware of every vulnerability. But I do expect that a financial institution has a specialist somewhere that audits the code before it is sent for testing ("white box"), and then I expect them to have an independent audit team probe for vulnerabilities ("black box").

After the inexperienced developer has had his code rejected for various flaws, he will become quite aware of the obvious ways things like this can go wrong.

Don't get me wrong, I expect that the vast majority of developers wouldn't make this mistake in the first place. But if you aren't a specialist, it is pure hubris to think that you write code that is hardened against all of the attacks out there. And if you have a vulnerability, it really doesn't matter if it's an embarrassingly simple vulnerability or one that requires sophisticated techniques to uncover and exploit. Either way, you're road kill.

High-value targets like banks need two security specialists (the code audit and the penetration test) to accompany the development specialists. That's simple separation of concerns, and it works as well in team organisation as it does in code organisation.

raganwald2• 15 years ago

Unrelated anecdote:

When working with a substantial financial organisation, there were strict rules prohibiting my team from communicating with the code audit and other security teams. Going for coffee with one of the internal audit people would get you both fired. The external testing was done by a security consulting firm. Their reports were passed on to me with the identifying details stripped out so that I wouldn't even know who they were.

Talk about "low coupling!"

tptacek10• 15 years ago

It shouldn't matter to you who the external firm is. Stripping out their contact info, at least in the archives, is probably smart; it makes it easier for the company to keep multiple security firms working on their projects. Which, when you're a huge bank, is what you want to be doing.

biot• 15 years ago

No doubt the goal there is to prevent collusion between a developer who introduces a subtle but exploitable bug and the auditors/testers who might be interested in failing to find such a bug, for an appropriate percentage of course.

tptacek10• 15 years ago

I don't know anyone at Citi, but I do know several firms that do audit & pentest work there, and I'm told it's high-volume --- in other words, it's likely every app gets tested. This is a startling miss.

When you engage a software security firm to check out an app, you're trusting that their report is reasonably complete. As a practitioner, this is basically your nightmare scenario. You know in the back of your head that it's always possible you're going to miss things, but a vulnerability this trivial seems like the kind of thing you find on the first day of testing.

joe_the_user4• 15 years ago

This is the kind of vulnerability that should be caught on the whiteboard before a single line of code is written.

I know this isn't what you're talking about, but I hope that an institution that's protecting my money will formally specifying their protocol and not haphazardly putting their app together with "Test Driven Development". But maybe that's how they managed to be this stupid.

keenerd• 15 years ago

Citi fails on a lot of other basic common sense practices. For example, their password fields are limited to 8 characters.

joe_the_user4• 15 years ago

Hmm,

Perhaps the problem is the "security specialists".

I agree that just about any reasonably skilled developer knows to, at the absolute minimum, include a user-specific session key with every transaction.

But I've also seen "security specialists" get a pass for a variety of bad behaviors. And we've heard of the complete incompetence of HBGary. My hunch is that the secrecy/obscurity that the passed for security in pre-Internet culture is exactly what allows total screw-ups like this to happen in these large organizations. The gold-plated consultants who are friends with the VP wind-up being far less competent than your average/decent developer.

tptacek10• 15 years ago

There's probably a kernel of truth to your comment, but that last graf makes no sense. The "security specialists" aren't writing the code with vulnerable direct object references; the "average decent developers" are. It's true that there are consultants out there that don't improve the (poor) seaworthiness of typical developer output, but few of them are so bad as to make the situation worse.

And, the notion that "good" developers turn out uniformly secure code is delusive. We're lucky to have gotten to work with some of the best run, most carefully recruited teams in the industry. You find terrifying things on all these engagements.

My advice, again as a practitioner: if your security firm isn't finding terrifying things, at least on the first go-round with an app, ask very tough questions. Ask if they need an extra week or two to catch up and get real findings. Then ask which of the two of you should pay for those weeks.

We are still finding code execution on a good chunk of our web gigs, even at the "good" companies.

InclinedPlane• 15 years ago

Sadly the problem is at a higher level than that, and not restricted to security. Today software development might as well be magic. There are no reliable objective methods for determining developer or software quality. The only method that's proven to work well is the subjective judgment of another "magician". The same applies to software security as well.

This creates a boot strapping problem. If you're an organization of magicians then you have a comparatively easy time finding other good magicians. But if you don't have that hsitory and you still have the need you won't be able to know the quality of people you're hiring except on the least granular of scales (lots of failed and low quality projects). This is a big reason behind so much of the problems of software development in "the enterprise". Because people who know how to run a bank or a media company haven't the slightest clue how to run software development, and there generally aren't trusted 3rd parties that people can go to for help.

Worse yet, there are perverse incentives at play. If you can't tell the difference between a good security person and a stunningly mediocre one, you are going to go with the cheapest one who you think is good enough (generally: solid job history, resume fully buzzword compliant, etc.)

lifeisstillgood2• 15 years ago

>>> Today software development might as well be magic. There are no reliable objective methods for determining developer or software quality. The only method that's proven to work well is the subjective judgment of another "magician".

Try replacing software development with writing english.

I am going to stop banging on about this eventually, but reading and writing code is something that needs to be understood at all levels of a company before that company gets good at it. If the boss of the company and everyone who reports to him is illiterate, one cannot expect the memos and the policy documents and the amusing posters to be of any decent quality

TheCondor• 15 years ago

I've been ranting on this stuff for a couple years now to my friends. There are some alarming trends.. First off, a pen test is often treated the same as an attorney client relationship. If the test turns up particularly costly bad news, I've seen a handful of testers have the relationship essentially severed, received some hard language about talking about it from a lawyer and then received a check from a private account as if the company doesn't want to leave any traces that they actually knew about the problems. (I'm not joking, some medium sized companies have done this)

With some of the regulations the big missing piece is openness, there is no transparency into it at all. Any audited company should say who audited them and then after some period of time, 180 days maybe, the audit should be made public. The business risk is that customers will leave, in many cases like Playstation Network, customers effectively can't leave, they've already invested in something and there isn't an alternative. In many other cases it's not typically going to be widely publicized. If the customers can't leave, en mass, there is no business pressure for security and without any transparency the regulations will simply be gamed.

evro• 15 years ago

I was reading about this yesterday and about an hour later, Citi called me trying to sell me their fraud protection. I replied "Did it help the 200,000+ accounts that were stolen from you?" and hung up.

khafra• 15 years ago

It might have--I mean, when Citi sells you fraud protection for your Citi account, that's basically insurance against either you or them being careless with your account info.

earlyriser• 15 years ago

I discovered that my bank, Banque Nationale, used GET to delete transactions from the History. Then somebody could send a mail to the bank clients with an image linked to this Get action and delete the transactions of the client if he was logged into the bank and reading his emails at the same time. It wasn't a big risk, but I don't understand how this went live. I mean if a bank could not get that POST is for C_UD and GET for _R__, then who?

jcoder• 15 years ago

``security experts'': The attack was "especially ingenious," and "would have been hard to prepare for."

The experts are certainly part of the problem.

yaix2• 15 years ago

I have really problams believe that this could be true. Not even a first year student would be that stupid to expose any user id in the URL. And read from it without access right checking. For access to the related account data. How would they even get the idea to do such a thing?

And as for the "hackers", I guess legally this was not even a break-in. At least in Germany, for legally being a break-in, a computer system must be "specially secured with the intention of preventing access". Well, this system wasn't.

...still, I have a hard time believing that it could be true.

prodigal_erik2• 15 years ago

There's nothing wrong with having a user id in a URL. That's very RESTful (my account is distinct from your account, so they really ought to have different URLs), and can make a lot of sense in cases where users may delegate permission to manage each others' records. It's just trusting that user id without an authorization check that's idiotic.

yaix2• 15 years ago

Its RESTful yes, but with the wrong item of data.

To identify a logged in user and give the user access to their private account data, ONLY EVER use a unique and temporal random string. Nothing else. Ever.

Storing that random string in the URL may be done but is more insecure, because it will remain in the browser history. Not good if the user in on a public PC. Better store it in a cookie.

enobrev• 15 years ago

I personally prefer NOT to use Primary Keys in URLs for this exact reason. At the very least you can generate unique and difficult to guess keys that correspond to those IDs. A record ID (user id for example) has no use outside of database joins and therefore is useless in the public domain (url). As well, for public data (obviously not bank data) a data-based-key (song name, restaurant title, whatever) is far more useful in debugging and in logs.

Otherwise - you're absolutely right allowing access to protected records - via id, key, or whatever - should absolutely be checked against the authenticated session upon every request.

stef252• 15 years ago

wouldn't a simple IF( $url_id !== $logged_in_id ) die("No access") fix this?

slavak• 15 years ago

If they saved the $logged_in_id in the session then they wouldn't need to pass it as a parameter in the url. No, this probably indicates their entire architecture is ass-backwards.

defdac• 15 years ago

The same bank also complains if you want to use more than 9 characters in your password.. That kindof hints about how they store passwords in their database..

runningdogx• 15 years ago

Sadly, other financial services companies do not seem to understand, either.

Etrade's password system requires 6 to 32 characters with at least one number according to the password change instructions. They don't mention punctuation, but if you try to include some, they're deemed to be "invalid characters". Go figure.

This stuff makes me sad that I signed up for etrade:

"Thank you for your message regarding enabling HTTP Strict Transport Security. I have sent a request to our Product Development Team to have this feature added. Due to the high volume of requests, there is no guarantee that this will be implemented." -- etrade representative

"ETRADE does not allow certain characters to be used when establishing a password online. There is no specific ETRADE publication providing information on why certain characters are not allowed, such as special characters used for punctuation, etc. We appreciate your feedback concerning the online passwords. I forwarded this suggestion to the Product Development Team for future consideration and implementation. I can not guarantee when or if this change will be able to be made..." -- another etrade representative

anty• 15 years ago

Did anyone else check if the article was meant to be satiric? This is unbelievable!

jdp23• 15 years ago

Historically, companies like Citi haven't faced any meaningful consequences for putting their customers at risk by not doing "security 101". Will things be any different this time?

Arxiss• 15 years ago

How the hell could this happen? I think that 6yr old kid and figure out URL injection... wait a second... citi is a bank???

bskari• 15 years ago

What worries me the most is that one expert that the Daily Mail interviewed said "It would have been hard to prepare for this type of vulnerability." The same expert "wondered how the hackers could have known to breach security by focusing on the vulnerability in the browser."

http://www.dailymail.co.uk/news/article-2003393/How-Citigrou...

gnosis3• 15 years ago

I wonder what Citibank is going to do about this now? Are they going to change their customers' account numbers?

That's the minimum that should be done for those accounts which were compromised.

sixtofour2• 15 years ago

I've never worked for a bank, or any company that held sensitive information. I've only worked for companies that sold products to be used internally. Grains of salt are on the table to your left.

What this looks like, in the context of all the other serious recent breaches like Sony and the IMF, and from the point of view of someone who's never had to fight this particular battle but knows a little code, is that these corps deployed online apps in the early days when this wasn't a major part of their corporate face. Practices and points of view evolved from an initial environment where there just wasn't as much motivation for criminals to crack apps, because there wouldn't be that much of a market for what they stole. So corps could get away with deploying almost anything, relying on both security through obscurity and security through rarity (breaches were rare due to low profit). People in corporate offices that even knew their corps had these apps would be rare because the prestige of managing these people and apps would be low.

The apps we have today would then be direct descendants of the old insecure apps, and in many cases would be built directly on those old apps. Layers of mud, and you can't change the inside layers because old mud is brittle.

And now the corps are going up against, not people who are merely exploring or looking for bragging rights, but people working for criminal enterprises that, while not having the global scope of banks, are large enough and focused enough to directly challenge the technical power of the banks. And the banks are working with old, dry mud.

Again, grains of salt, but I suspect I'm in the right salt mine.

Sukotto• 15 years ago

I use citibank for my family accounts so may have been impacted. Has anyone posted a list of compromised account numbers someplace I can check against?

nandemo• 15 years ago

> Think of it as a mansion with a high-tech security system -- but the front door wasn’t locked tight.

It's more like an apartment building with high-tech locks in the front door and apartment doors. But after you unlock the front door, you can unlock any apartment's door with your key! The keys are all identical, only the number printed on the label is different.

teyc• 15 years ago

I wouldn't necessarily blame this on the guy programming this. However, the person who spec'd the application up would be due for a quick demotion. The problem with antiquated bank systems is that the teller is trusted with the access to any account. So when it came to web enabling the old teller application, someone did some screen scraping as a prototype without having a concept of restricting access.

There is probably no concept of linking an authenticated account to a restricted set of bank accounts. Instead, they've probably wired it up to CICS directly to retrieve account details. This is why the Quick Fix appears to be obsfucating the account number in the URL.

Is there a public report anywhere? Aren't companies required to report all privacy breaches?

shaydoc• 15 years ago

Its a basic error. Tech Architect should be seeing this in milliseconds. Its a total design flaw. Should not be going straight to SQL with just paramters in a querystring, there should at least be authenticated user account verification checking.

Also doesn't say much for the company doing security review, its a basic check. Furthermore to not have a user/onwer id to join on there (no doubt sql back end) is shameful. I mean I can see it now :

select x from accounttable where accountnumber = @val

how about simply :

select x from accounttable where accountnumber = @accno and ownerid = @ownerid

How Hackers Stole 200,000 Citi Accounts Just By Changing Numbers In The URL

Related Stories