PostgreSQL 9.2.4, 9.1.9, 9.0.13 and 8.4.17 released

http://www.postgresql.org/about/news/1456/

by edwinvlieg • 13 years ago

252 92 13 years ago

facorreia• 13 years ago

"This update fixes a high-exposure security vulnerability in versions 9.0 and later. All users of the affected versions are strongly urged to apply the update immediately."

dkulchenko• 13 years ago

So if I have no databases that start with "-", I'm not vulnerable? Didn't quite understand what they meant by that.

sgift• 13 years ago

Just from the quote cited by octo_t I would read that you are still vulnerable: A malicious database user could craft a _connection string_ which contains a database name starting with -. There's no hint that the database has to exist on your server for this to work, so I would read it could be a complete bogus request and still damage your files.

qompiler• 13 years ago

/* Is this all it takes? */

PQconnectdb("host=127.0.0.1 dbname=-exploit user=postgres password=postgres port=5432");

jsaxton86• 13 years ago

From the FAQ originally shared by edwinvlieg, you are still vulnerable:

The vulnerability allows users to use a command-line switch for a PostgreSQL connection intended for single-user recovery mode while PostgreSQL is running in normal, multiuser mode. This can be used to harm the server.

masklinn• 13 years ago

Nope. Looking at the release notes:

> Fix insecure parsing of server command-line switches (Mitsumasa Kondo, Kyotaro Horiguchi)

So I assume command-line switch parsing is somehow involved in parsing the connection string (probably because the same connection strings can be used from API and from CLI?), I guess a database name with a leading `-` can be interpreted as a switch and execute corrupting commands.

edit: according to the dedicated FAQ:

> The vulnerability allows users to use a command-line switch for a PostgreSQL connection intended for single-user recovery mode while PostgreSQL is running in normal, multiuser mode. This can be used to harm the server.

octo_t• 13 years ago

This is the main vulnerability I presume

> A connection request containing a database name that begins with "-" may be crafted to damage or destroy files within a server's data directory

I just. No words.

r4vik• 13 years ago

defense in depth though, users shouldn't be able to craft connection requests to begin with

SoftwareMaven• 13 years ago

Then how would a user use the db? Not every use of a database is behind a web application.

phillmv• 13 years ago

Getting this stuff right is hard. Don't be a hater.

Just because the attack vector looks simple doesn't mean the bug was obvious.

16s• 13 years ago

I agree. Postgres is one of the most well thought out DBs I've ever used. They are slow to add features but when they do, they are done right with lot's of attention to detail. Everyone makes mistakes.

gcr• 13 years ago

Absolutely. Remember the MySQL authentication bypass vulnerability¹, where a blank password would succeed to authenticate 1/255th of the time? This reminds me of that.

1: http://thehackernews.com/2012/06/cve-2012-2122-serious-mysql...

__alexs2• 13 years ago

The FAQ makes no mention of this massive data loss possibility which seems a bit odd…

jeltz4• 13 years ago

The exploit does not actually destroy the files. It allows appending data to the files making the server crash. The file could be recovered by simply removing the junk data added to it.

From the FAQ:

> Persistent Denial of Service: an unauthenticated attacker may use this vulnerability to cause PostgreSQL error messages to be appended to targeted files in the PostgreSQL data directory on the server. Files corrupted in this way may cause the database server to crash, and to refuse to restart. The database server can be fixed either by editing the files and removing the garbage text, or restoring from backup.

danielweber3• 13 years ago

If you have never released software with a security hole, it is because you have never released software.

Very smart and conscientious people can mess things up here. Shaming serves no purpose.

edwinvlieg• 13 years ago

More information about the security release can also be found in the special FAQ:

http://www.postgresql.org/support/security/faq/2013-04-04/

Bootvis2• 13 years ago

I'm not an expert so I'll ask here:

Is there an attack vector if you run PostgreSQL locally, no untrusted users are able to create connection strings and do not allow remote access?

It seems to be no but I prefer to be sure ;)

mark_l_watson• 13 years ago

I would like an answer to this, quickly, also. I am scheduled to leave on a 6 hour hike in one hour - I have time to update if I have to. I only permit localhost connections.

talkingquickly• 13 years ago

From the FAQ:

Who is most at risk:

"Any system that allows unrestricted access to the PostgreSQL network port, such as users running PostgreSQL on a public cloud, is especially vulnerable. Users whose servers are only accessible on protected internal networks, or who have effective firewalling or other network access restrictions, are less vulnerable."

So looks like it's low risk but they're not willing to say no risk.

Intermernet• 13 years ago

Do it, it involves about 3 seconds of downtime per server to run sudo aptitide update and sudo aptitude upgrade . Other package management tools should be equally speedy. If you've rolled your own postgresql binary, get the new sources, re-build, stop services, replace binaries, start services (the old drill...)

The upgrade (on Debian based systems at least) is for libpq5, postgresql-9.1 and postgresql-client-9.1 . You shouldn't need to do anything else unless, for some strange reason, you actually do have a database starting with "-".

Also, I wish you a fine hike!

(probably commented too late, but I hope you had a nice hike, nonetheless)

gingerlime5• 13 years ago

as far as I understand from the FAQ page, as long as connections to your PG database are blocked from external sources, you should be safe. Seems like a good idea to upgrade as soon as possible anyway though.

  > How can users protect themselves?
  > * Download the update release and update all of your servers as soon as possible.
  > * Ensure that PostgreSQL is not open to connections from untrusted networks.
  > * Audit your database users to be certain that all logins require proper credentials, and that the only logins which exist are legitimate and in current use.

EDIT: added a quote from the FAQ for clarity.

buttscicles• 13 years ago

It doesn't specify whether pg_hba.conf is sufficient to protect against this, anybody have any word on that?

rmc• 13 years ago

To be sure, you should upgrade. Why do you not want to upgrade?

protomyth• 13 years ago

[not gp post, but this is a normal situation]

Because some folks need to go through a long, drawn out process to upgrade internal software and might already have some place in the pipeline to put the upgrade so it will get tested with everything else.

If its a security issue for your current installation, you go now and hope, but if it can wait for a release then you do that.

Bootvis2• 13 years ago

Thanks all for the information.

The reason I wanted to not upgrade directly is that access was hard at the moment of posting. Upgrading should never wait too long but this was (luckily) defcon 1 for us.

vacri• 13 years ago

It's more that bootvis has no immediate slack time to deal with issues if the upgrade isn't smooth.

rst• 13 years ago

A firewall might be adequate; configuring postgres itself seems not to be. The vulnerable code is invoked before client authentication, so anyone who can make a tcp connection to the postmaster process can exploit the attack, even if their source IP would otherwise get them unconditionally bounced.

fulafel2• 13 years ago

you don't need a firewall as long as you don't turn on remote connections. that's the listen_address option.

gingerlime5• 13 years ago

"Heroku was given access to updated source code which patched the vulnerability at the same time as other packagers. Because Heroku was especially vulnerable, the PostgreSQL Core Team worked with them both to secure their infrastructure and to use their deployment as a test-bed for the security patches, in order to verify that the security update did not break any application functionality. Heroku has a history both of working closely with community developers, and of testing experimental features in their PostgreSQL service."

I believe all the heroku hosted postgresql servers are externally accessible and there's no way to filter access by IP.

Of course hindsight is always 20:20, but perhaps it's a good idea for heroku to consider adding some basic (optional) firewall layer to allow customers to control who can connect to the hosted db?

Disclaimer: I'm not a heroku customer. I did however consider moving our pg's over to them a little while ago.

thibaut_barrere3• 13 years ago

I would definitely pay Heroku a bit more to restrict access of a Heroku PG instance to a given EC2 security group.

I couldn't find a PG hosting provider with more useful features than Heroku - with that, it would be a killer option.

mryan3• 13 years ago

It certainly seems feasible with the use of VPC and Security Groups.

At the very least I think they should offer an option that only you to restrict access so only your dynos have access to the postgres port.

sfjailbird• 13 years ago

I would love to have Heroku add this web interface to the default PostgreSQL package: http://teampostgresql.herokuapp.com

gingerlime5• 13 years ago

yes, when I did my little research for hosted PG, heroku was pretty much the only viable option. That said, I did come across some difficulties running `rake spec` against the heroku hosted db (since you can't drop the database, only individual tables). This was giving me some (unrelated) headache.

Another thing I was really hoping for but couldn't find with heroku, was being able to do a point-in-time-restores via the heroku web/cli interface. This would be a seriously nice feature if something like that was available...

fdr2• 13 years ago

Well, I'll tell you why it is not implemented that way. I hope the restriction can be lifted some day. I am a member of the Heroku staff related to the matters at hand.

The problem is the sheer number of Heroku Runtime machines which are located in a smattering of IP space, and rapidly and accurately propagating the firewall rules required for tight network access control as applications churn around in there...even then, there have been some reports of voluminous firewall rules causing obscure problems. Of course: the world is an obscure place and yet we can deal with it in time. Such a thing could be hardened I'm sure, but the amount of bookkeeping required is a bit terrifying, and experience suggests that will not go entirely smoothly or be easy to find bugs in. At the time this was reasoned out (maybe about two years ago?), it wasn't even widely known that Heroku offered any data storage service of significance. Early days.

So, the simple approach is to enable access from the entire Heroku Runtime layer. But who can put applications there? Anybody on the Internet. That's why the 'ingress' feature to poke a temporary hole in the database firewall was dropped as too marginal given the fairly severe inconvenience of it all -- it had the feel of a weird Heroku-ism, especially in light of the lack of attacks using unauthenticated clients on Postgres, until this date. In addition, what about all the other addons? There would need to be an API, and because of the nature of what it is doing (poking holes before beginning, say, TLS negotiation which requires even more round trips and is slow enough as it is) it would need to be stable, fast, accurate, and all that other good stuff, lest all addons effectively be rendered offline all at once.

Other more application-level approaches are possible (like tunneling all connections to a local unix socket, or something), but that's but a little strange, because it requires code injection of strange stuff into the running container, makes your URLs look funny, and so on. This model has been experimented with by some of my colleagues and staff of other, similar firms, and definitely has its attractive sides. Nevertheless, one of the general guidelines in our implementation choices is to not be too weird for someone moving on or off the service. These approaches are also not in and of themselves immune to DoS or security problems...they need careful auditing. Maybe another look is indicated. And again: what about every other addon? What about your local computer? Are you going to install some weird agent, open source or no, from every such addon?

My personal favorite pie in the sky option right now would be cordon off a slice of contiguous, publicly addressable (but not publicly accessible) IP space so that firewall rules could remain compact and slower changing, and also involve your local computer: imagine being able to VPN to two or three such networks simultaneously, because their addressing does not alias at all. But this is still in the realm of fantasy, and and probably would require looking at IPv6 to be able to segment the address space in a sane manner, which compounds another layer of stuff that can be buggy, even in such mundane details as address parsing.

So....there's some rambling, but I thought it perhaps useful to talk about some of the challenges here to motivate the discussion.

ipsin• 13 years ago

I'm a little confused about their release strategy. Perhaps someone can explain it to me.

They took their repositories private to secretly develop the bug fix. Then they released the fixed versions along with what seem to be enough details to trigger the bug for anyone who hasn't patched.

Sure the patch contains the same information in source form, but if they'd gone light on details while saying "seriously, go get this", there'd probably be fewer curious vandals trying to delete your database while you're reading HN.

mryan3• 13 years ago

I like to know exactly why I'm updating my database before I apply any patches. I doubt they could have been sufficiently light on the details, while still giving admins enough information to decide whether or not to upgrade.

"Apply this patch, don't worry what it does, just do it" is not something I want to hear from my database vendor :-)

Had the repos remained public, this detailed information would have been available to a lot more people, a lot sooner. Temporarily "going dark" to work on the patch seems like an acceptable compromise.

sophacles• 13 years ago

Not really, any big project has people going over every commit to see what changed. Any commits that are associated with a security release are particularly scrutinized. Within an hour of release there would already be people talking about the vulnerability, as well as example code for triggering it. Full disclosure is better, because even if people can't do an upgrade, they can choose to block ports at firewalls, turn off databases, and other mitigation methods immediately, as they are allowed to.

Hiding the information just weakens the defender position, not the attacker position. Secrecy in implementation is not security, it is just stupidity.

keypusher• 13 years ago

Take a lesson from open source, security through obscurity does not work. Better to be fully transparent and honest about the flaws and their fixes, and get the word out there so that people update their boxes quickly.

mkopinsky3• 13 years ago

While this comment is wrong, it does not deserve the downvotes that it's gotten. The guy asked a reasonable question, now let's be polite and answer it (as this comment's sibling indeed do). Downvotes should be reserved for comments that undermine productive discussion.

danielweber3• 13 years ago

If they were closed source, they could probably get away with it, buying hours to days of time before someone reverse-engineers the attack.

They are open source, though, and many people who use it build from source. It is very very easy for complete amateurs to look through the source and see what changed in a manner of minutes.

timdorr• 13 years ago

The commit that fixes it with a few more details: http://git.postgresql.org/gitweb/?p=postgresql.git;a=commitd...

    An oversight in commit e710b65c1c56ca7b91f662c63d37ff2e72862a94 allowed
    database names beginning with "-" to be treated as though they were secure
    command-line switches; and this switch processing occurs before client
    authentication, so that even an unprivileged remote attacker could exploit
    the bug, needing only connectivity to the postmaster's port.  Assorted
    exploits for this are possible, some requiring a valid database login,
    some not.  The worst known problem is that the "-r" switch can be invoked
    to redirect the process's stderr output, so that subsequent error messages
    will be appended to any file the server can write.  This can for example be
    used to corrupt the server's configuration files, so that it will fail when
    next restarted.  Complete destruction of database tables is also possible.

    Fix by keeping the database name extracted from a startup packet fully
    separate from command-line switches, as had already been done with the
    user name field.
    
    The Postgres project thanks Mitsumasa Kondo for discovering this bug,
    Kyotaro Horiguchi for drafting the fix, and Noah Misch for recognizing
    the full extent of the danger.
    
    Security: CVE-2013-1899

ashray2• 13 years ago

I'm really happy that the PostgreSQL team was able to fix this so quickly and it does appear to be a massive security issue. However, on the flip side, in 13+ years of web development work, I've never really seen a database name beginning with "-".

ceejayoz• 13 years ago

I don't think you have to have a database starting with - for the bug to work.

manojlds• 13 years ago

This bug wouldn't be such a big deal if such a name was a requirement for the exploit.

danielweber3• 13 years ago

Can someone follow up on this? Do you need the db name to already begin with "-" to exploit this?

EDIT: No, the problem is in parsing, not the existing names: https://news.ycombinator.com/item?id=5492508

themckman• 13 years ago

That is a fantastic commit message. I try and convince anyone I work with that they should write commit messages like this. Slowly, but surely, they come around.

sickpig2• 13 years ago

And the author is a fantastic coder/developer, really!

https://en.wikipedia.org/wiki/Tom_Lane_%28computer_scientist...

calpaterson• 13 years ago

Ubuntu repos already seem to have the fix

instakill• 13 years ago

Should one update the local postgres version? Any write ups on how exactly to go about it?

selenamarie4• 13 years ago

Yes. http://www.postgresql.org/docs/9.2/static/upgrading.html , second paragraph:

PostgreSQL major versions are represented by the first two digit groups of the version number, e.g., 8.4. PostgreSQL minor versions are represented by the third group of version digits, e.g., 8.4.2 is the second minor release of 8.4. Minor releases never change the internal storage format and are always compatible with earlier and later minor releases of the same major version number, e.g., 8.4.2 is compatible with 8.4, 8.4.1 and 8.4.6. To update between compatible versions, you simply replace the executables while the server is down and restart the server. The data directory remains unchanged — minor upgrades are that simple.

simon_kun• 13 years ago

It's testament to Canonical that Ubuntu 8.04 LTS still gets security patches backported to 8.3. If you (still) have servers running Hardy, it's 'apt-get upgrade' time: http://www.ubuntu.com/usn/usn-1789-1/

joevandyk2• 13 years ago

Sorta surprised I don't see people complaining that this release contained other changes besides the security fix.

Lots of folks complained about that (unintended) ActiveRecord change in Rails during the last security update.

PostgreSQL 9.2.4, 9.1.9, 9.0.13 and 8.4.17 released

Related Stories