Knusperli: A deblocking JPEG decoder

by gsnedders • 8 years ago

121 44 8 years ago

nneonneo2• 8 years ago

Ah, this is a great idea, and I can see how this could be extended beyond simple deblocking.

In essence, it computes the uncertainty from the JPEG quantization step (the source of most of JPEG’s compression) and uses that information to select a decoding that minimizes a particular error metric (here, discontinuities at block boundaries).

But it’s not limited to just that, in principle. It could optimize towards any decoded block within the quantization range given a suitable prior - and I imagine that this could be paired with e.g. a DNN to faithfully reconstruct an image from a JPEG with much better accuracy.

lokopodium• 8 years ago

The same should be applicable to MP3s. Also, the continuity measure could include derivatives as well.

blt• 8 years ago

yeah, the deblocked image looks a little softer. Makes sense, the optimization objective encourages it. Sometimes the block boundary aligns with an image boundary. Agree that incorporating a DNN could be a cool solution.

sitkack• 8 years ago

Could a DNN be thought of a continuous decompression dictionary?

psandersen• 8 years ago

Additionally, I'd love to see a decoder add noise proportional to the uncertainty and surrounding structure as over smoothed images look unnatural as well. See activity masking demo https://people.xiph.org/~jm/daala/pvq_demo/

Ruud-v-A2• 8 years ago

Pik can synthesize noise. https://github.com/google/pik

disposablename• 8 years ago

DNNs are not magic, and not everything needs them, especially a fucking JPEG decoder, come on.

bwang29• 8 years ago

I think visually speaking, while the deblocked version is smoother overall, the blockiness added extra perception of noise/texture. The extra noise almost always increase the perception of sharpness and level of details, and you can see from the braids coming out of the hat that it does look sharper with the blocking version, although the wall is smoother. Would be great to include more sample images in the page.

niftich• 8 years ago

x264 video codec developer Dark Shikari blogged about [1][2] this "fake sharpness" phenomenon. Early H.264 encoders produced excessively smooth, deblocked video, which made them look worse compared to MPEG-4 ASP (like Xvid), despite former being a newer, objectively better format. Wavelet-transform compressors like JPEG2000 had a similar problem vs. blocky JPEGs.

People's tastes are subjective, though, so it's good to have options.

[1] Archive.org: http://web.archive.org/web/20141124213159/http://x264dev.mul... [2] Archive.is: http://archive.is/K7jKA

lawl3• 8 years ago

Unacceptable. Even though swiss-german doesn't have any spelling rules, I hereby declare a public vote to deport google from switzerland, because it's clearly spelled "Chnuschperli" in proper swiss-german.

Edit: Sorry, I forgot HN is allergic to jokes, obviously this is turning into reddit. I apologize.

nneonneo2• 8 years ago

So what does Knusperli even mean? It seems to mean “crunchy” or something from a Google search but Translate turns up nothing. Is the meaning at all related to the project, or is it just a cute name?

lawl3• 8 years ago

Yes, it means "crunchy thingy" basically. The name doesn't have any relation to the project. It's small pieces of fish fried in a dough mantle.

I was jokingly complaining about the spelling because the previous names "Brötli" and "Zöpfli" of similar projects were 'properly' spelled. They don't have anything to do with the name either. "Brötli" and "Zöpfli" are basically types of bread.

While Knusperli is a very german way of spelling a swiss german word. Though, it is what you would find in a supermarket.

Yes, too obscure of a joke, sorry :(

justinwp• 8 years ago

Seems a new library could use something besides Lenna...

funkaster• 8 years ago

I thought that by this point in time, we would have more images than Lena to showcase image libraries.

vanderZwan3• 8 years ago

After all, no image compression format will ever beat LenPEG as long as the she is the standard

http://www.dangermouse.net/esoteric/lenpeg.html

gixo• 8 years ago

Agreed, I believe the image of Lenna should be discouraged, as some people might find the image offending and unwelcoming to women (it originally appeared in a 1972 Playboy magazine issue).

https://en.wikipedia.org/wiki/Lenna

danieltillett2• 8 years ago

Considering Lena doesn't object and by all accounts is quite happy her image is used, there really is no reason not to use her image. If you real want you can use Fabio instead.

tinus_hn• 8 years ago

Brace yourself, the social justice warriors have arrived!

xmichael99• 8 years ago

C# wrapper? :D

divbit3• 8 years ago

I'm not sure if you are joking, but it does seem that more interoperability is usually a good thing.. (especially for something like a jpeg decoder - maybe it's less useful to have a toaster interface smoothly with some coffee beans)

divbit3• 8 years ago

Although I guess coffee roasters are a thing..

anameaname• 8 years ago

Dissenting opinion: I think the blocky version looks better when not zoomed in. The sample on the left is less smoothed, and doesn't look as airbrushed. It looks better when zoomed in, but not when are regular zoom.

gok• 8 years ago

Curious how this affects visual quality indexes like SSIM.

(Also, come on, it’s 2018, we need to retire Lena)

mrob• 8 years ago

See also https://github.com/victorvde/jpeg2png , which tries to maximize smoothness everywhere instead of just at block boundaries.

vardump2• 8 years ago

Maybe these two approaches, jpeg2png and Knusperli, could be combined to get potentially superior end result.

Knusperli would run first, to reduce quantization discontinuities a bit.

TD-Linux• 8 years ago

There is a lot of previous art on this sort of thing, like the ffmpeg spp filter: https://ffmpeg.org/ffmpeg-filters.html#spp-1

It's unclear to me if this method will perform better than those methods.

Animats• 8 years ago

Nice. This is a decode-side thing; it can be used on existing images.

But the site should show more images. Some with hard edges, like text and line art, need to be included.

vanviegen• 8 years ago

Why would that be interesting? Jpeg is the wrong format for these types of images anyway, is it not?

vardump2• 8 years ago

That's precisely why it would be interesting. Getting cleaner decodes of jpeg-encoded line art and screenshots would be great.

Some images are only available as badly encoded jpegs.

IshKebab• 8 years ago

Yes but there still isn't a universally supported "good format" so JPEG often gets used for text.

vanderZwan3• 8 years ago

Because it might still help if someone else made the wrong choice in image format.

rasz2• 8 years ago

Test image still has perceptible blocking. Description suggests it should be possible to reconstruct source values/smoothness staying in stored intervals, and yet there are still visible boundaries.

Lower right corner you can clearly trace 4 horizontal blocks, looks like algorithm tries hard to maintain encoded hard boundaries despite there being gradual color shift. Maybe it needs to look further than direct neighbor block for clues.

tialaramex• 8 years ago

I would assume that JPEG like most lossy compression algorithms is specifically defined in terms of the correct uncompressed output for a given compressed input. This means a "deblocking JPEG decoder" is effectively actually just a bad decoder that gives the wrong output (albeit for some outputs maybe you happen to prefer this).

The definitions are this way around because it's the only way to have an opportunity to improve encoding quality over time, if you start by defining how encoding must be done then you lock in bad ideas. Lame for example is far, far better at encoding MPEG audio layer III than the demo encoders shipped when the standard was written, and this is only possible because the standard says how to turn MP3 back into PCM data, not how to encode PCM data as MP3.

Ruud-v-A2• 8 years ago

That is what I expected as well, but it turns out not to be the case. See also https://www.ece.cmu.edu/~ece548/hw/lab3/faq-do~1.htm and https://encode.ru/threads/2922-Knusperli-%E2%80%94-a-better-....

vanderZwan3• 8 years ago

So could one define an "inverse" JPG standard that works with the current version but has the advantages that tialaramex spoke about?

black_puppydog• 8 years ago

This is one of those nice "let's do this properly" projects google does. I'd put into the same category as their work on compression. Unsexy and "so last decade" at first glance, but decidedly useful and universally applicable, given the number of jpg files created since the dawn of time.

That being said, I'd really wish people would stop using lossy formats for everything. I can live with it for simple icons and such (although, why not svg?) but photos? come on! Of course, that would have to come with a nice connection for everyone.

Oh and this is definitely going to make for some head scratching for devs who assume image decoding is somehow deterministic. :D

rasz2• 8 years ago

their work on compression

Like that time they stole Jarek Duda ANS and are still trying to patent it fraudulently claiming to have invented it 4 years after Jareks public domain publication?

https://encode.ru/threads/2648-Published-rANS-patent-by-Stor...

Knusperli: A deblocking JPEG decoder

Related Stories