Show HN: Backblaze Reed-Solomon erasure codes

https://github.com/Backblaze/JavaReedSolomon

by budmang • 11 years ago

16 5 11 years ago

Nice. For those wondering what Reed-Solomon codes are all about: they are extra data that allows certain types of corruption to be detected and fixed.

I believe RS codes are used on optical media such as CDs and DVDs which allows perfect playback with a few scratches here and there. They're also used in QR codes, so even if the scan is imperfect, the data can be reconstructed correctly. In the case of QR codes, the standard allows the QR code creator to set the level of error correction, providing a wide range of of data size vs resilience to corruption choices.

budmang• 11 years ago

Exactly. Thanks for adding that. They're also used in nearly all large storage systems from Amazon S3 to RAID6.

m-app• 11 years ago

And also for Forward Error Correction (FEC) on Optical Transport Networks.

nacs• 11 years ago

Reed-Solomon is also used in parity files (Parchive [1]) which were especially popular in the newsgroup/usenet days.

parity files could be created that were then uploaded along with the original data files. If any of the data files were damaged or lost while being propagated between Usenet servers, users could download parity files and use them to reconstruct the damaged or missing files.

[1] https://en.wikipedia.org/wiki/Parchive

orasis• 11 years ago

In 2000, I wrote a Java reed solomon implementation that also includes native libraries for linux-x86 and win32. I expect it to be at least 5x faster than a pure Java implementation.

http://www.bytebucket.org/onionnetworks/fec

Show HN: Backblaze Reed-Solomon erasure codes

Related Stories