Security Lessons Learned From The Diaspora Launch

http://www.kalzumeus.com/2010/09/22/security-lessons-learned-from-the-diaspora-launch/

by pw • 16 years ago

250 138 16 years ago

I've always wondered about this (not being a code-monkey-ninja-wizard, myself)...

If you open source something, unless it's perfectly written, wouldn't the hacking potential be... near 100%? If everyone can see how you do everything it seems like even a minor slip up will potentially surrender your site.

Could someone explain this (I'm probably missing a piece of the puzzle I can't place)?

tptacek6• 16 years ago

On about 80% of our projects we'll never see source code. Not having source code is a speed bump for a professional appsec tester. The problem with Diaspora isn't that it's open source; it's that they launched their open source project when it wasn't ready for that.

patio116• 16 years ago

Having source code makes both attacks and defenses easier. It isn't automatic that a bug present in source code will be caught, either by attackers or defenders -- "a million eyes make all bugs shallow" is basically horsepuckey. The bugs in the PNG processing routines, for example, took something like a decade for someone to find and exploit. All Java implementations of OpenID are OSS. All are vulnerable to timing attacks, at least as of a month or so ago. I implemented one of them and passed my implementation past our resident God King of Engineers and he didn't see that fault, either.

Now, if you're a highly anticipated project and you're making errors covered in every Security 101 article which happen to be very visible, then OSSing your code makes it highly likely that people will see those, for good and ill. What scares me for Diaspora's future isn't those errors -- it is the part of the iceberg below the waterline. I mean, if you're steaming at full speed towards a gigantic "I'M GONNA RIP UP YOUR BOAT!" sign, there is probably something underwater and I doubt any qualified security guy (I am so not one) will donate you a few tens of thousands of dollars to tell you how screwed you are right now.

T_S_• 16 years ago

The idea is that people can help you eliminate insecure methods--if the code available. "Security through obscurity" leaves you with no assistance and engenders a false sense of security where determined attackers are concerned.

That said, MPWILGSIANSE (my password is LadyGaga so I am no security expert)

ElbertF• 16 years ago

That's why you don't run pre-alpha code on a production server. The advantage of publishing code this early is that people like the author can point out flaws and help secure the software.

bl4k6• 16 years ago

With source code available attackers are white-box testing as opposed to black-box testing.

Black-boxing is where you throw known arguments at a system and measure the responses, and hence deducing what the system is doing

White-box testing is knowing what is happening internally, so you can immediately skip to step two of a security test - which is exploiting.

With black-boxing it takes a very very long time to learn the entire system and its workings, but it can be done. Having the code just means skip that test. For eg. with Diaspora from it launching to it being exploited was a matter of minutes.

The Twitter URL escaping bug from this week was from within one of their public source code repositories. While they don't release everything as open source, they have released enough to give an attacker a good view of their stack and how it works. Bugs not detected in the open source code are likely to also appear in other parts of the platform that are closed source (since they weren't detected in the first place)

tptacek6• 16 years ago

I can probably go all night on this, but a couple things from a quick read of this (very good) post:

First, mass assignment.

The answer to mass-assignment bugs is "attr_accessible". Accessible attributes can be set via update/build/new; nothing else can. Every Rails AR model should have an "attr_accessible" line in it.

I've met smart dev teams working under the misconception that attr_accessible means "these are the attributes that can be changed based on user requests", and so virtually everything is made accessible. No! If something's not attr_accessible, you just set it manually (user.foo = params[:user][:foo]). It's not painful and the extra line expresses something important ("this is a sensitive attribute"). Attributes are inaccessible until they prove themselves mass-assignment-worthy.

Second, the string interpolation in the regex.

Real quick: don't ever let users interpolate arbitrary strings into regular expressions. Regular expression libraries are terribly complicated and not very well tested. To illustrate (but not fully explain) the danger here, run this line of code:

     ruby -e "'=XX===============================' =~
     /X(.+)+X/"

There are worse things that can happen to you with regex injection than a trivial DoS, but that should be enough motivation.

Oh, one more thing: I appreciate Patrick's take on systems failures breaking Rails apps before underlying crypto flaws will, but even if they had protected their keys, their crypto wouldn't have worked. Don't build things that require crypto. You aren't going to get it right.

patio116• 16 years ago

Every Rails AR model should have an "attr_accessible" line in it.

I'd do you one better: use an initializer to monkeypatch ActiveRecord::Base and fire "attr_accessible nil", which will cause mass assignment to fail on any object you create from a class which doesn't make the assignment explicit.

tptacek6• 16 years ago

That's clever. Want a job? =)

poet3• 16 years ago

I'm keeping this in my back pocket the next time I have a conversation about why I prefer Ruby's monkey patching paradigm to Python's strictness. This is better than all my current examples. :)

patrickaljord• 16 years ago

In Shapado we use a safe_update methode like this so we always need to specify which attribute can be updated:

@question.safe_update(%w[title body language tags], params[:question])

jerf2• 16 years ago

I'd go one further and say that if you use string interpolation with anything remotely related to user input at all in a web app, you probably just wrote a security hole of some sort. I use "probably" in the frequentist sense of the term, because the odds of a given user-input string interpolation in an arbitrarily-chosen web app being in at least one of 1. an HTML string 2. a database string 3. a javascript string or 4. something else that is executable by something, without the context-relevant encoding function, are quite high. Same goes for any string concatenation involving anything from the environment or user without the conspicuous presence of an encoding function of some kind, since interpolation is just syntax sugar for concatenation.

"Don't let users interpolate, ever" is close to truth. It isn't quite truth, but it's a lot shorter than the truth.

djhworld2• 16 years ago

I don't have access to a box with ruby on at the moment, what does that line do?

maximilian• 16 years ago

It just hangs ruby (like in an endless loop)

mike-cardwell2• 16 years ago

ruby -e "'=XX===============================' =~ /X(.+)+X/"

Why does that hang in Ruby? In Perl it's fine...

avar3• 16 years ago

Because Ruby uses PCRE, but Perl uses its own regex engine which handles cases like that a lot better. That pattern is rejected right away by Perl's engine because it sees that there isn't another X in that string:

    anchored "X" at 0 floating "X" at 2..2147483647 (checking floating) minlen 3 
    Guessing start of match in sv for REx "X(.+)+X" against "=XX==============================="
    Found floating substr "X" at offset 2...
    Contradicts anchored substr "X", trying floating at offset 3...
    Did not find floating substr "X"...
    Match rejected by optimizer

Changing the pattern to /X(.)X/ will cause Perl to do real work. But it'll complete it in 0.003s while Ruby will just hang.

I wrote an article a while back about the differences between PCRE and Perl's engine: http://use.perl.org/~avar/journal/33585

code_duck10• 16 years ago

Well... basically, it sounds like Ruby's regex engine needs some work, hmm?

jharrison2• 16 years ago

Not to be totally nitpicky but if they're using any recent version of Rails (I haven't looked at the source yet), the DESTROY action doesn't respond to GET by default. That doesn't change the fact that they don't scope deletes to the logged-in user's assets.

tptacek6• 16 years ago

POST vs. GET is a little bit of a red herring anyways, since either method works for CSRF. (I'm adding to your comment, not amending it).

jharrison2• 16 years ago

I'm certainly not qualified to argue with you on anything security related (nor would I want to). I'm simply clarifying Rails default handling of destructive actions which is probably semi-offtopic.

bl4k6• 16 years ago

"For example, if you were logged in to a Diaspora seed and knew the ID of any photo on the server, changing the URL of any destroy action from the ID of a photo you own to an ID of any other photo would let you delete that second photo."

When I was working as a pen tester I would completely scold developers for letting this happen - telling them that with everything we know today about security and good programming practices there is no way you should allow that to happen. Off by-one bugs, timing attacks etc. are more excusable, but this, this is just amateur hour.

That was 11 years ago.

llimllib3• 16 years ago

> That was 11 years ago.

The problem is, these kids are from college. They don't teach you stuff like "writing a secure web application" in college, or even try to.

(Not that this is unreasonable, though perhaps I'm suggesting that there should be different career paths for CS majors and people who intend to be professional programmers. (I say as a CS-educated professional programmer))

bl4k6• 16 years ago

They likely didn't learn Rails in college either

jlgbecom4• 16 years ago

I've always thought it would make more sense for CS degrees to be for computer scientists (ie, people who want to do more high-level theoretical work), and that software development was more of a trade school, where you learned the languages, and were soon thrown into real-world style projects and apprenticeships.

Imagine if your nurse came out of college having never stepped foot into a hospital, having only read about how to take vitals and such, but never having done it on a live human being.

rb2k_5• 16 years ago

> They don't teach you stuff like "writing a secure web application" in college

We had a class on it. They basically pushed us through OWASP from front to back :)

aidanf• 16 years ago

Any decent rails book will have a section on common security flaws and how to avoid them. There are plenty of web tutorials on the same topic. All the flaws in the linked article are very basic and should have been avoided if they took the time to RTFM.

code_duck10• 16 years ago

If you need a college professor to tell you "Check that the resource is owned by the logged in user requesting a change before you change it", you might never, ever be a good programmer no matter what. This was apparent to me when I'd been doing PHP for like 6 months. These issues are common sense.

DannoHung• 16 years ago

I disagree, this should be an obvious security capacity: Don't let people who are not permissioned to modify a given resource modify a given resource.

I might be able to excuse this since they're fundamentally still in alpha (or pre-alpha) and were rushing to get code out.

danieldon• 16 years ago

My first web app had authorization. I didn't even go to school for CS.

Simply put, they focused too much on trendy tools and libraries like MongoDB and CarrierWave and neglected the basics.

makmanalp• 16 years ago

What a stupid excuse. I'm a "college kid". I don't neglect simple access control in my software.

photon_off• 16 years ago

First of all, this type of thing, preventing users from just changing around URLs, shouldn't need to be taught. It is pretty common sense. When you make something like this, if you don't wonder: "Gee, what would happen if somebody changed photo_id=123&delete=true to photo_id=124&delete=true, would it delete photo 124?" then I'd have to say you aren't a very curious individual. That likely doesn't bode well for your programming prowess.

Validating user input is probably the first thing you learn about web application programming, which is frequently taught at universities, or in books titled "web application programming" which you should at least skim if you're going to start a project like this. Don't blame college for this. Just because it is something that isn't focused on in college (it is, though), and they went to college, does not mean it was college's fault. Would it be fair to blame college for any other mistakes they made, just as long as college did not "focus" on it? No. Some things are common sense.

Most likely, the culprit was time constraints, which is far more excusable.

naner• 16 years ago

The problem is, these kids are from college. They don't teach you stuff like "writing a secure web application" in college, or even try to.

Bingo. I went to Georgia Tech, which has a pretty damn good CS program, and I had to hunt for security classes. One was a "special topics" course that wasn't available very often and didn't have anything to do with application security (was a Net. Sec. course). The other was not a CS course but a Comp. Eng. course and was focused on penetration testing. :/ I actually earned a "Network Security" certificate with my degree which I never even knew was available (it wasn't mentioned anywhere in the course literature).

Since I've graduated they have redone the whole CS dept so I don't know if things have changed, though.

But like someone else said, a lot of this stuff is common sense, especially if you're a programmer and have systems knowledge. And I think most programmers have the habit of imagining all the different ways things could break when they are coding, too. Like a hackers curiosity that most of us share. I know when something looks obviously wrong on a website or in an application I'm using I start to poke around and see what I can uncover.

tszming• 16 years ago

Would appreciate if more articles like this are posted on HN, useful and practical!

llimllib3• 16 years ago

For that to happen, HN update speed would have to slow dramatically.

(I favor this and agree with you, incidentally.)

grovulent• 16 years ago

Yeah - I usually come on here to find the technical sorts of articles that I learn from and have been seeing less and less of these lately.

techiferous• 16 years ago

It's true. And as a result, I'm seeing more and more threads like this one. :)

bl4k6• 16 years ago

"NoSQL Doesn’t Mean No SQL Injection"

I lol'd. Mind if I use that?

MongoDB is harder to secure and filter because you have all of Javascript to worry about, rather than just SQL (and where most servers can escape arguments themselves through prepared statements etc.).

SQL databases are also well understood (for eg. in MS-SQL I can stop the remainder of the statement from executing with '--'). MongoDB with its JS engine is still a big unknown.

najirama• 16 years ago

Similarly:

"...secret squirrel double-plus alpha unrelease..."

Mind if I use that? It would be a terrific title for an animal fighting game I've been itching to make.

KaeseEs• 16 years ago

It's a snowclone of a line from Animal House.

Tichy• 16 years ago

I haven't used MongoDB, but I think there is a fundamental difference in the way data is updated. I don't think you have to escape JavaScript in the user input, because you don't update by submitting a single String to execute. The user input is just data.

bl4k6• 16 years ago

It depends on the client driver. They have insert, delete, save etc. which send those commands with the user supplied data encoded, but most of the drivers also have an exec or execute which dumps what the user enters straight onto the db.

for eg.

http://www.php.net/manual/en/mongodb.execute.php

"This method allows you to run arbitary JavaScript on the database."

mrkurt• 16 years ago

Most Mongo queries don't involve javascript, they're abusing a special operator.

rubyrescue• 16 years ago

I like that the phrase "Lil Bobby Tables" to reference dropping the database via SQL injection has entered the hacker vernacular.

What would be a nice one-page security guide would be a 'lil bobby tables' guide to databases - SQL injection for any database - (SQL or NoSQL) - the goal being to help developers prevent these attacks.

jerf2• 16 years ago

Well, I don't have a ready reference for SQL injections, but how about one for XSS? http://ha.ckers.org/xss.html If that doesn't teach you anything, well, my hat's off to you.

kree10• 16 years ago

It exists: http://bobby-tables.com/

gfodor• 16 years ago

I'll admit, when people came down hard on them vis a vis security, I figured it was just a bit sloppy as a preview release.

I was wrong.. these aren't really security "holes" as that's not strong enough a word. I think the best way to put it is they accidentally created the first social network wiki.

pjscott• 16 years ago

When you put it that way, suddenly I'm intrigued by the concept.

mechanical_fish• 16 years ago

Obviously real people will object when you manipulate their friend lists and their photo galleries. But there's real possibilities in a wikified social network for fictional characters.

code_duck10• 16 years ago

Maybe they should take that strength and build on it! Then they can say they were visionaries instead.

I agree though, these aren't like subtle security holes that would need a security expert to review. Checking that a user own the resource on which they are requesting modification is basically common sense.

jodrellblank• 16 years ago

You've said this twice now, and it still isn't true. Don't be the Youtube commenter who, on a video on how to hold and what to call parts of a musical instrument and how many people got it wrong, said "this video is useless, it's all common sense".

If it were common sense to do it, they would have done it. It's not. It's a very distinct thought pattern shift from "the browser is a part of the execution of our code and it will only try a delete link which the code has generated" to "the user can request anything at any time no matter what links we have or haven't generated or what they can see on screen".

It's a learned shift specific to some subsets of some kinds of computer programmers, not at all "common sense".

(and besides, even if it were common sense, what's the point in your comment then?)

aberkowitz• 16 years ago

I knew Diaspora had problems when I got an email from the guy who hosts Openspora telling me that [two] other people had the same username.

shajith2• 16 years ago

Thanks for the web-app security pointers in the post: the RoR security guide[1] and the OWASP Top 10[2]. These should be required reading for anyone making a public web application.

[1] http://guides.rubyonrails.org/security.html (Well-written, like the other guides. Totally worth reading fully).

[2] http://www.owasp.org/index.php/Top_10_2010-Main (Open Web Application Security Project's top application security risks for 2010)

jlgbecom4• 16 years ago

There are a lot of lessons to be learned from Diaspora, beyond the (brilliant) points made about security.

1. When media hype provides you with $200k, you're still best served by bringing people's expectations down to earth. There was no way they were going to be able to build anything approaching a Facebook killer in 3 months, and it would have been best if they would have made that clear in the beginning.

2. There are a number of projects like Diaspora that have been working for years towards the same exact goal. The only way Diaspora could have succeeded where those have failed (or not-yet-succeeded), is if they had properly articulated where they would go right where others had gone wrong. If you can't do that, you probably don't have the perspective necessary to take on such a huge project.

3. We all need to be less susceptible to the story of the "boy wonders" taking on the establishment. Between Diaspora and Haystack+, these should be sobering lessons about the dangers of hype and a good human-interest story when the result we need is stable, well-written code.

+ http://blog.jgc.org/2010/09/myth-of-boy-wizard.html

4. Open source isn't magic. Rails isn't magic. Having a good idea and a whole lot of heart isn't magic. It's true that the software world isn't exactly a meritocracy, but at the same time, we need to recognize that you have to be able to build something generally usable, and if you can't, there's nothing that will save you except for harder work and learned lessons. Some bugs will be fixed by the interest in Diaspora, but there are big architecture questions here that need to be resolved, and coordinating that democratically through the internet in a sea of strangers is a logistical nightmare. And while rails does provide a lot of functionality out-of-the-box, a project of this size isn't held up by how long it takes to write the photo-uploading code, it's held up by the big-picture stuff that rails can't really help you with. In the end, programming is programming. You need specs, mockups, user stories, documentation, all kinds of unsexy stuff.

I think we'll probably see Diaspora stabilize into something usable at some point. But I'm very doubtful that will be anytime soon, and I'm especially doubtful that it will be before the other projects (Elgg, Appleseed, OneSocialWeb, StatusNet, etc) mature into the facebook killer people want to see.

Building the kind of open source social networking software necessary to take on Facebook at it's own game is such a massive, complex undertaking, and it's such undiscovered territory, that there really is a big disadvantage to being the new kid on the block.

bl4k6• 16 years ago

Good post. It comes down to inexperience, and I am sure we have all been through it ("pfffft.. I could do that in 4 hours" -- younger me).

There were red flags in their kickstarter post:

* "We are four talented young programmers from NYU’s Courant Institute trying to raise money so we can spend the summer building Diaspora"

* "Diaspora knows how to securely share (using GPG) your pictures, videos, and more."

* "We have a plan, a bunch of ideas and the programming chops to build Diaspora. What we need is the time it takes to iron out a powerful, secure, and elegant piece of software. Daniel, Ilya, Raphael, and Maxwell are all ready to trade our internships and summer jobs for three months totally focused on building Diaspora"

* "We promise to you that Diaspora will be aGPL software which will released at the end of the summer."

They also said that no similar system exists (they do)

ericb• 16 years ago

> Open source isn't magic.

Isn't it though? I think popular open source projects have magic. These guys put out a demo with lousy unsecured code and rookie mistakes and within a week the worst offenders were identified and repaired.

I'm not saying they are great developers, but certainly OSS is some sort of powerful magic.

jlgbecom4• 16 years ago

Open source is good at getting the small bugs, the syntax errors, the security holes with clear and direct fixes, etc.

Open source is not good for making use of the "Many eyeballs" for architectural decisions. It's where the wisdom of crowds provides little benefit. This is why most open source projects are not a haphazard bazaar of stone soup contributors, like most people think, but are actually small dedicated teams of talented software developers, who are usually paid.

I think our conception of what OSS can achieve, simply by being OSS, is somewhat inflated.

(Don't get me wrong, though, OSS is fantastic, and I think it's, by design, much better than close source)

jeremymcanally• 16 years ago

One more bug in the first code snippet is their use of find_by_id means that @album could be nil, causing an error when they attempt to edit it. Most people would use @album.find and then catch the error/show a 404.

I realize these guys are in college, but they really should have (a) brought people's expectations in line with their abilities and (b) reached out to experienced developers to help them out. Intridea probably would have given them a few developer hours per week to help code, advise them, and so on. Just a little input and guidance would have saved them a lot of grief.

code_duck10• 16 years ago

Totally, their lives would be so much better now if they had just asked 2-3 experienced Rails developers to review their work before releasing it like this. Actually, the fact that they didn't think to do that kind of illustrates a problem.

josegonzalez• 16 years ago

They shared a space with Pivotal Labs. Either they were lazy and didn't think to ask about stuff that matters - ui is nice[1], but securing your app is more important - or the Pivotal Labs guys dropped the ball big time.

I'm inclined to think it was neither and they just didn't think anyone would notice. It happens.

[1] http://www.joindiaspora.com/2010/07/01/one-month-in.html

equark2• 16 years ago

I don't think this is that big of a deal. Pretty much every developer I know has learned about security through this exact process. Either a senior developer or user exposes the flaw and smart, but new, developers quickly realize the didn't understand the attack angles. Without concrete experience it's pretty hard to appreciate how exactly these attacks work. But after a few exploits you start getting paranoid, understand the basic problems, and are always thinking and reading about how your code might be missing something. These guys seem reasonably smart so I expect after a few months of exposure, they'll be fine.

code_duck10• 16 years ago

Why would the community of experienced developers who are supposedly expected to be interested in working on this project, find it rewarding to sit around and wait for them to work through their training wheels? This whole situation really is absurd.

equark2• 16 years ago

It doesn't seem absurd to me that a few college friends want to hack together a project to fix a problem they see. I think it's great, regardless of whether they succeed. Nobody is forcing you to join them.

dzlobin• 16 years ago

The biggest problem I think, and I speak from experience, is that some of the best rails tutorials leave out a lot of important security issues entirely. Some of them might do a trivial paragraph on password encryption, but most of them leave out any mention of these kind of basic security flaws.

Even if the writers didn't explicitly write chapters dedicated to security, obviously even that wouldn't be enough, they should at least note which code is entirely unsafe in production, and where you can learn more about it.

Every rails book is filled with this sort of code that people learn, and then use.

jasonkester5• 16 years ago

Obligitory reference:

http://expatsoftware.com/articles/2007/03/examplecode-produc...

(exampleCode != productionCode)

pkulak• 16 years ago

Wow, those are some pretty amateur mistakes. Just taking whatever data you got from the user with no checks? Really? And that's not something you fix later. That should stick out like a sore thumb the moment you write it, making you check that user == logged_in_user before you even move on. Wow.

robryan• 16 years ago

Excellent article. I can see why under the kind of time pressure they have been under there would be issues like these.

Interesting I was thinking in comparison to PHP where a beginner would much more likely be individually assigning each input into a SQL statement. Sure there are all kind of possibilities that can go wrong there to but mass assignment looks like it makes it so easy for someone to shoot themselves in the foot.

rb2k_5• 16 years ago

After reading the article, also read this comment: http://www.kalzumeus.com/2010/09/22/security-lessons-learned...

The comment negates some of the statements made in the post

jasonkester5• 16 years ago

No, that comment just helps to explain how trivial these things would have been to work around. That's the whole point of the article, that these guys missed all the most obvious things that you need to do to secure your application.

There are no deep, tricky issues explained because absolutely zero effort was needed to find a half dozen breathtakingly bad practices floating at the surface.

So yes, of course it's trivially fixable. The problem is that it wasn't trivially fixed and they thought they were ready to release it.

rb2k_5• 16 years ago

Please read the whole comment I linked to. I was talking about things like the OP saying that browsers could delete things by prefetching (GET requests) or that update_attributes does a double assignment which are simply not true

zeemonkee2• 16 years ago

It's not that the pre-alpha Diaspora has insecurities that bothers me, it's the whole execution.

What I really would like to see is a documented protocol - based on XMPP or some other established, well-tested protocol would be good, but if not then at least something.

Once you have that protocol - which tells you how Diaspora "seeds" communicate securely - you can let others build their own implementation, using Rails, PHP, Python, doesn't matter. Sure, release a reference implementation in Rails, but the protocol is the most important thing.

Unfortunately what we have is just another Facebook clone done in Rails, which is disappointing.

arethuza3• 16 years ago

It looks like a classic case of poorly managed expectations - the technologists would prefer an approach like the one you outline (and this was what I was expecting), however given their visibility they had to deliver a working application that people could download and install and have it do something.

While meeting both of those objectives in those timescales might be possible it would be a truly remarkable achievement. Not surprisingly it didn't happen and they released something that pleased nobody - all we can hope for is that they learn some lessons and move onto better things.

woodson2• 16 years ago

That's what I found interesting when looking into OneSocialWeb, their focus on already existing protocols (XMPP plus some XEPs http://onesocialweb.org/developers-xmpp.html), instead of adhering to Not invented here.

andrewf• 16 years ago

Google tried this with Wave.

Nobody stepped up to write a decent client, and the product was judged (unfavorably) on the merits of the reference implementation.

jasonkester5• 16 years ago

Lesson learned: Never let the outside world see your First Big Project Ever.

This is what Fred Brooks would have called the First System. Everybody builds this thing at the beginning of their career, and it's always this embarrassing. Mine, in 1996, took this a step further and actually prepared SQL statements in javascript before submitting them to the server to run. Yours probably did something equally bad. It's the rule of the First System. It's where we learn all the lessons we need so that we can do things right the Third time (I'll spare the Second System description for the time being...)

The key though is to make sure that nobody ever sees that code. Hopefully it will be locked away in some intranet vacation-time-planning app that nobody will ever dig into. That way you can look back at it in shame, but few others will ever know about it.

So here we have a team of people who have clearly never built anything at all, trying to learn on the job while being scrutinized by the entire world and actually submitting their code for public review.

God help them.

arethuza3• 16 years ago

Yes, but Fred Brooks also warned that the Second System would be even worse!

jasonkester5• 16 years ago

No need to but. I actually mentioned that above :)

bradleyland• 16 years ago

Quite the contrary. Provided they have the right attitude, the programmers at Dispora just learned more on their first project than they ever would have if they hadn't shown it to anyone. That's one of the primary benefits of participating in open source software.

jasonkester5• 16 years ago

I think the point is that this is the stuff that you always learn in the course of building your First Big Thing. By opening it up to public ridicule rather than to a polite code review by an understanding senior dev, they don't really get any extra learning out of the deal. Just extra humiliation.

Tomek_• 16 years ago

I had my doubts about Diaspora, but now I know for sure: it will be a complete fiasco. Those are horrendous errors that show that those guys have completely no idea about web programming - and I can't see them learning it quickly (certainly not before planned release date of the final version). Sad thing.

code_duck10• 16 years ago

Indeed, the level of errors shown in this demonstrates that they would probably need about 2-3 years of experience to be decent at doing work like this. I think the best thing they could have done is hired 3 skilled developers to work for 6 months at $5k a month.

zeemonkee2• 16 years ago

Which begs the question - where did the money go ?

What they've released looks like your average weekend github side-project. I suspect a large % of Hacker News members have projects like this (though hopefully with better security ;-)). So what did they spend the money on ?

chegra2• 16 years ago

I appreciate the two links with how to handling security issues in web application.

As many have pointed out, they do not teach this stuff in University and acquiring such knowledge you tend to have to be very proactive about your development if you do not have industry experience.

So thanks a lot for sending these nuggets our way.

You stated that the team is manifestly out of their depth in terms of web security, how do you suggest that they proceed given their month deadline? Can they pay a security expert to resolve these issue or no one wants to come within a mile of this?

patio116• 16 years ago

So one of the painful lessons I picked up in industry is this: an impossible deadline plus a great desire to change is still an impossible deadline. Scheduling is not my bag, and I'm far less well acquainted with Diaspora than you are, but if I was four man months from release at the day job with the state of the project where I think it is I would be sending out emails saying "We will not hit this. It is impossible. We need to cut scope or push back release." to my superiors.

I don't know if you can get a security expert to fix this for you. You can certainly wave a big enough check around to get somebody to look over your code, but that won't magically improve code you haven't written yet. Also, it is highly likely that Diaspora is architecturally insecure -- that, beyond the "Oops, didn't check the input" code-level whoopsies that your federation strategy as written (and apparently as not documented outside of the sourcecode) just cannot be made to work right.

chegra2• 16 years ago

Thanks for the reply. I agree with the impossible deadline bit. Can you give an example of what you would consider a architecturally insecure web application?

batasrki• 16 years ago

That's what I feel. Not only do they have an impossible deadline to hit right now, but the deadline they did hit was probably impossible as well.

They may have spent most of the summer architecting this thing and figuring out things like "seeds". When it came down to putting code on screen, as it were, they had minimal time to do so. All of these security issues feel like the release was rushed and that they might have been better off releasing it silently, without a way to deploy public nodes, and having a blog post explaining the situation.

bherms• 16 years ago

This is a great post and shows how awesome of a community we have. The Diaspora guys are a bit behind when it comes to solid programming, but Patrick (among others) reviewed the code, noted flaws, reported and advised them on how to fix them, then wrote a great post explaining what was wrong to help others avoid the same mistakes. Stuff like this is both educational and helps the programming community move forward. Thanks to Patrick.

10ren2• 16 years ago

What idiots those Diaspora guys are! All that excellent security consulting, for free! They don't know the first thing about software development! It makes me so mad, I'm going to write up a carefully researched and detailed account of other errors they've made! Then they'll see how clueless they are - again! Ha, what amateurs!

10ren2• 16 years ago

EDIT here because too late to edit my above comment I just want to clarify, to distance myself from the recent genius/tragedy submission: I'm mocking the people who criticize the Diaspora guys for not being perfect. I'm pro release early, release often and anti perfectionism. IMHO, Patrick is doing exactly the right thing - helping to build something better. But my post could be interpreted as mocking Patrick - that's not what I meant, and I apologize for the ambiguity.

On reflection, I shouldn't have expressed this through mockery at all, but through helping. :(

c00p3r• 16 years ago

OK, you've made headlines. Now, where are your patches? ^_^

agentultra• 16 years ago

Did anyone really believe that a bunch of guys were going to write a secure privacy-based facebook-killer application in three or four months?

That's a challenging proposition even for experienced teams.

Though we all know that they warned us that the software was still full of bugs and treated as experimental.

Patrick says that doesn't matter, that they haven't got the foundation right and anything built on top of it will likely fall.

Maybe they can still iterate and fix those things. Maybe they will scrap large sections of the code and re-write it properly. I suggest they keep trying and learn something from all of this. Us older geeks can be pretty harsh sometimes. We often expect new-comers to not make the same mistakes we did once. That's how we learn though, so don't take it the wrong way.

Next time just don't promise more than you can deliver.

jmspring• 16 years ago

It is funny, I have looked at the code and am not a big ruby/rails person, but see obvious flaws and security holes. However, a lot of the discussion around these have been either in the "it is amateurish" bucket or the "are they expecting help from the OSS community? that is leveraging eyeballs, that may not come".

Now, while it hasn't been their primary intension (most likely), to have the OSS community (and others curious about the code) provide fixes and feed back, they sure have gotten a lot of input and advice. I also suspect they have learned quite a lot through the feedback - positive, constructive, and inflammatory. You really can't ask much more -- aside from some direct help.

While I have no plan to run my own Diaspora node, I do look forward to seeing how the code and project evolve.

adamdecaf• 16 years ago

Well, I for one am glad that I learned from articles like this. It's a hell of a lot easier to read discussion from "real programmers" and learn from them. Now that I'm in college stuff like this is second nature.

Security Lessons Learned From The Diaspora Launch

Related Stories