Vitess: Scaling MySQL databases for the web

http://code.google.com/p/vitess/

by 1010011010 • 14 years ago

101 16 14 years ago

code.google.com

enneff3• 14 years ago

I'm glad to see this released because it is great technology that showcases Go's strengths. Here's why they chose Go, in their own words:

"Go is miles ahead of C++ and Java in terms of expressibility and close in terms of performance. It is also relatively simple and has a straightforward interaction with Linux system calls."

It has always frustrating to say "We use Go at Google, I just can't say how." So I'm happy that now I can say that YouTube use Go in their core infrastructure, serving more than a billion requests per day with this service alone.

foobarbazetc• 14 years ago

How does this help "scale" databases?

It just looks like a connection pooler with a RPC interface?

Am I missing a feature?

enneff3• 14 years ago

The feature list on the linked page lists a lot more than that. Also see:

http://code.google.com/p/vitess/wiki/ProjectGoals

tcard• 14 years ago

A quick glance at the sources shows immediately what Go offers per se in terms of readability and maintainability. Go strongly encourages, even forces at compile time, certain style constraints that results in such a delightful code to explore. Were it written in C, you'd have to deal first with arcane #whatever directives in .h and .c files and strange makefiles and such stuff.

I hope someday Go will get the acknowledgement it deserves as a real piece of genius. I really encourage recommend just to take a 15-min try at http://tour.golang.org.

canop_fr3• 14 years ago

The thing would probably look better if adapted to Go1, without all those makefiles and deps.

dkhenry2• 14 years ago

What I don't understand is why you would use MySQL as your storage engine it seems to me you are stripping away most of the benefits of using MySQL by interfacing it with this library. Is there a compelling reason to not use google's own LevelDB?

mattyb• 14 years ago

Is there a compelling reason to not use google's own LevelDB?

LevelDB is a key-value storage library. MySQL is a relational database server. They are about as different as data stores can get.

stevencorona2• 14 years ago

Don't really see what parts of mySQL you are stripping away by using this library. It's just extending/improving some weak spots of mySQL- you literally don't lose any features.

dkhenry2• 14 years ago

According to their website - """ The way we plan to achieve this is by providing a subset SQL front end with limited guarantees """

If your making something to parse and validate SQL yourself which is what it seems like they are doing then why go through all the hassle to run it a second time through another SQL parser. Also since your mandating everything have a "Key" why are we not using a "Key-Value" store.

I am not saying this is a bad project or a bad design decision I am just curious why they made those decisions since at first glance it makes no sense whatsoever.

stevencorona2• 14 years ago

This project looks awesome and really points out some of the major weaknesses with in mySQL/libmysql-

  * Impossible/difficult to gracefully handle connection/query timeouts.
  * Extremely poor query/row cache (the table's entire query cache is flushed anytime a write is made in the table)
  * Poor connection memory usage- Each open connection sucks up ~10MB of memory

harlowja• 14 years ago

Odd, so instead of helping fix the underlying libraries they created yet another layer. Sadness++

jakejake• 14 years ago

At a glance this seems fantastic. I have an application where the database recently started to get overloaded at peak usage times.

It seems like this could be dropped in as a replacement with very little code re-factoring if the client library is similar to the native mysql client lib. (or an adapter could be written)

Does anybody know what would be involved in getting this working with PHP?

canop_fr3• 14 years ago

Interesting comments (in mysql.go) :

    // Muahahaha
    func strlen(str *C.char) int {
            gstr := (*[1 << 30]byte)(unsafe.Pointer(str))
            length := 0
            for gstr[length] != 0 {
                    length++
            }
            return length
    }

BTW, is Google planning to make a proper mysql driver in go (implementing database/sql API and not unsafe) ?

enneff3• 14 years ago

Here's a list of database/sql drivers: http://code.google.com/p/go-wiki/wiki/SQLDrivers

canop_fr3• 14 years ago

Yes, that's the driver I'm using. But my question was related to the fact that Google seems to have made (see source) a partial mysql driver, so I was wondering if they were planning to make it database/sql compatible.

haarts• 14 years ago

It is rather unclear to me whether or not Vtocc automagically takes care of sharding and replication.

Vitess: Scaling MySQL databases for the web

Related Stories