CSI: Compressing Similar Images

https://news.ycombinator.com/item?id=40600992

by barfbagginus • 2 years ago

1 2 2 years ago

news.ycombinator.com

I've got a 20 gig database of business pictures in JPEG format - they're closeup pictures of boat decks from 700 different boats. However many boats are similar, and many images share identical pixels from slightly different views. So I'm wondering if I can compress them with a differ compression engine that better exploits the similarity and overlap between images. I'm also wondering if I could fine tune a neural network image compressor.

What tools would you use in production? Or is this still experimental territory? And if so, which projects should I invest in?

opendomain• 2 years ago

I have done extensive work on compression.

I have a solution that I may be able to modify to meet your requirements.

contact me at zip AT web3 DOT net

barfbagginus• 2 years ago

It seems a bit shady, with respect. I want to talk about papers and publicly available systems that are out there. I'm not going to engage with an amoral web 3 head who's got some proprietary code in his trench coat and wants me to walk into the alleyway to do a deal. I'm building an open source system, so others can train their own databases, not just me.

Are you interested in publishing your code as an open source system? Then I will pledge at least 40 man hours to the code base, doing tasks that you've been too busy to take care of, but which will make your code and clients happy.

I have background in image processing and compression , python packaging/pypi deployment, CI/CD, testing, and benchmarking. I could definitely help you knock down some of your tickets.

What do you think? A little too brusque on the approach? Or a potential colab?