Show HN: EDB – A framework to make and manage backups of your database

https://github.com/RoxasShadow/EDB

by RoxasShadow • 11 years ago

31 8 11 years ago

github.com

dchest• 11 years ago

Filed issues related to crypto: https://github.com/RoxasShadow/EDB/issues/created_by/dchest

RoxasShadow3• 11 years ago

Thanks for the good points raised!

inquist• 11 years ago

Simple, clean, straightforward. I've done similar things in bash before but not bothered to publish :) It's a good idea though to have an ecosystem of simple but flexible tools like this.

Here's another idea we implemented alongside a backup system like this. Say you run a backup daily; you can run another script to prune the archives, keeping D most recent daily archives, W most recent weekly archives, M most recent monthly archives, and all yearly archives.

RoxasShadow3• 11 years ago

Thank you for your feedback.

That would suite well as module when used alongside the FTP module. I'll keep it in mind :)

mickeyben• 11 years ago

I've recently created a similar tool at our company.

We found out that dumping the database table by table in a parallelized way is much faster than a full database mysqldump.

There's also mysqlpump soon available in mysql 5.7 to replace mysqldump and works in parallel.

falcolas2• 11 years ago

Until then, have a look at "mydumper", which does both parallel dumps at a table level, but also chunking, which makes de-duplication possible.

falcolas2• 11 years ago

The mysql driver needs additional work. In particular, it needs a '--single-transaction' flag, or a global lock, to ensure that the dump is consistent - particularly if you want to dump multiple databases concurrently.

Doing one dump per table with chunking (each file has N rows) would help with both speed and disk sizes of backups by allowing S3 or some other program to implement de-duplication between incremental backups.

It also wouldn't hurt to capture the binlog position, if available, to enable point in time recovery.

Have a look at mydumper for an idea of how another tool implemented these:

https://launchpad.net/mydumper

RoxasShadow3• 11 years ago

I will open an issue with your suggestion. Thanks.

Show HN: EDB – A framework to make and manage backups of your database

Related Stories