Migrating Graphite from SQLite to PostgreSQL

http://obfuscurity.com/2013/12/Migrating-Graphite-from-SQLite-to-PostgreSQL

by craigkerstiens • 13 years ago

39 19 13 years ago

andrewvc• 13 years ago

As a fan of both databases, I just have to say, you don't always need all the constraints.

Also, at the very least, graphite should have been able to enforce them at the app level in addition to at the DB level.

obfuscurity_7• 13 years ago

I completely agree. I was rather disappointed to find that it didn't.

yeukhon6• 13 years ago

Is it by default it ships with SQLite or just Doc?

yeukhon6• 13 years ago

There is no reason why one would use SQLite in real production. Even during testing and dev, I am a firm believer of testing against real database that you are going to use in production. The problem "it's hard to setup a new database" is a result of not willing to invest in re-provisionable system.

untog• 13 years ago

There is no reason why one would use SQLite in real production.

Yes there is. Small storage for a local app? SQLite is ideal.

yeukhon6• 13 years ago

We are not on the same page. I was referring to Graphite and general web application. We are talking about real server serving pages to users.

Localstorage and app in desktop and mobile are definitely arguable.

obfuscurity_7• 13 years ago

Except that while Graphite is a "real web application", it's generally internal-facing only. As such, I never looked at it as needing a true "production-ready" RDBMS with real concurrency. And to be honest, we got by just fine with it in production at Heroku and GitHub, at heavy volume, running in a single server on SQLite. It never bit us, and all things considered, was a reasonable choice.

That said, knowing what I know now, I would've never done that. ;-)

yeukhon6• 13 years ago

I disagree because it is internal facing lol. If the software is capable of running on a true production-ready DBMS, there is no point why it can't be done already.

If SQLite is cheap to use because it's a single file, well, a sudo apt-get install PostgreSQL and then a few commands to set up the database and user password won't take you longer than 10 minutes. That's what I am arguing. There is no reason why we would use SQLite for heavy loading application such as logging. And I think we can now agree that's indeed true and I think we should promote people to use production-ready DBMS even during dev and testing, because those are the one you are going to use in production :)

mst• 13 years ago

There are plenty of reasons.

  * Single file to back up
  * Unix fs permissions sufficient for security
  * Simplicity valuable over write concurrency
  * Ability to write functions/aggregates/collations in the language you're embedding sqlite in
  * No need for maintaining and securing a system daemon

This article isn't, at all, about a problem with SQLite. It's about graphite shipping a stupid postgres schema. Had they used text for the column like anybody who knows how to use postgres would recommend, the graph names would've transferred fine.

TL;DR - graphite devs don't understand postgresql, graphite user misdiagnoses the problem and blames sqlite for it.

Someone• 13 years ago

The way I read this, this _is_ a problem with SQLite (somewhat). Both schemas limit the length of graph names, but SQLite ignores that part of the schema.

One can discuss whether it is a good idea to have that arbitrary, low limit, but that is not what this is about.

I agree that there are plenty of reasons to use SQLite in production. IMO, the conclusion should have been that one should run the same database in development, test, and production.

obfuscurity_7• 13 years ago

Excuse my french, but you're an idiot. I don't know if you simply didn't read the article, or are incapable of grasping the facts within.

alixaxel• 13 years ago

Meh, SQLite is awesome. So is PostgreSQL, but this rant is uncalled for.

obfuscurity_7• 13 years ago

How is the article a rant? Or are you referring to the one of the comments above?

Migrating Graphite from SQLite to PostgreSQL

Related Stories