The magic (image resampling) kernel

167 37 2 years ago

bhouston• 2 years ago

Super cool. How did I not know about this before?

I was also pretty surprised, as I consider myself decently knowledgeable in the field. Learned of it via https://github.com/libvips/libvips/issues/4089.

BoingBoomTschak4• 2 years ago

These previous discussions (including the author in the second one) were pretty fruitful:

https://news.ycombinator.com/item?id=10404517 (2015)

https://news.ycombinator.com/item?id=26513518 (2021)

pseudosavant• 2 years ago

I was surprised I hadn't heard of this, or his related project JPEG-Clear. I have thought for years that the JPEG-Clear method is how responsive images should have been handled in a browser. A single-file format that can be progressively downloaded only up to the resolution it is being displayed at. If you zoom in, the rest of the data can be downloaded for more detail. Doesn't require complex multi-file image authoring steps, keeps simple <img src> grammar, and is more efficient than downloading multiple completely separate images.

meindnoch2• 2 years ago

JPEG-Clear? The guy "reinvented" progressive JPEGs?

Dwedit• 2 years ago

Loading a more detailed version of an image as you zoom in is different from what a progressive JPEG does.

Loading a Progressive JPEG means you still unconditionally load the entire file, you just are able to show a low detail version before it is fully loaded. The last time I saw a progressive JPEG actually take time to load was when I had dialup.

svantana2• 2 years ago

I feel like this article is really overselling this filter. A 4-point symmetric interpolation kernel can be parameterized as [k, 1-k, 1-k, k]/2, i.e. it has a single degree of freedom. k=-1/4 is bicubic, k=1/4 is this 'magic', and k=0 is bilinear. Bicubic is sharper, and 'magic' has better alias rejection. Which looks better depends on the image and the viewer's subjective preference. For insta photos, it's probably better to go for 'magic', while for text, one might prefer bicubic. Neither is "simpler" as this article keeps suggesting, they just have different filter coefficients, that's all. But any other value of k is an equally valid choice.

BoingBoomTschak4• 2 years ago

It certainly is. Especially lacking in proper comparisons of the final filter with the competition. I myself default to RobidouxSharp for downscaling and something like https://www.imagemagick.org/discourse-server/viewtopic.php?t... for upscaling.

BoingBoomTschak4• 2 years ago

Took the time to make such a comparison using the article's sample images (even if the filter isn't sharpened in those), same thrice doubling of the small picture: http://0x0.st/XEEZ.png

I find such a test strange and irrelevant, though.

CyberDildonics8• 2 years ago

I guess anything is magic if you don't know how it works or if you need some clicks to promote your personal site.

This is basically a slightly different gaussian kernel and the "incredible results" of a small image becoming a larger resolution blurry image is completely normal.

Also you don't want negative lobes in image kernels no matter how theoretically ideal it is, because it will give you ringing artifacts.

If you work with image kernels / reconstruction filters long enough you will eventually learn that 90% of the time you want a gauss kernel.

pixelpoet• 2 years ago

If you work with image kernels / reconstruction filters long enough you will eventually learn that 90% of the time you want a gauss kernel.

Strongly disagree, and my commercial software is known for its high image quality and antialiasing. Gaussian is way too blurry unless you're rendering for film.

CyberDildonics8• 2 years ago

Then your filter width is too wide. Try 2.2 - 2.5

dahart8• 2 years ago

In my film experience, I think most film people don’t like Gaussian either; too blurry for them as well. At least, I’ve sat in on filter evaluations with a couple of directors & VFX sups many years ago, and they said Gaussian was too soft and preferred a sharper Mitchell. But I am curious, perhaps similar to the sibling comment - how do you determine the optimal Gaussian width? You can certainly go narrower/sharper and get less blur at the cost of more artifacts similar to a sharper filter, right? BTW have we discussed this recently? ;) I love Gaussian’s ability to hide any hint of the pixel grid, which I find very few filters can do. I also tend to believe that, perceptually speaking, over-blurring slightly doesn’t hurt while under-blurring does, especially for moving things, but that might be more personal bias than objective reality. I would be interested to look at any comparisons or results from your software or in general, if you have some.

DustinBrett• 2 years ago

Would be cooler if images on FB didn't suck.

rnhmjoj• 2 years ago

Fourthly, and most importantly, as noted above: m(x) is a partition of unity: it “fits into itself”; [...] if we place a copy of m(x) at integral positions, and sum up the results, we get a constant (unity) across all x. [...] This remarkable property can help prevent “beat” artifacts across a resized image.

So, basically the reason why this works better than other visually similar filters is that it happens to satify the Nyquist ISI criterion[1].

[1] https://en.wikipedia.org/wiki/Nyquist_ISI_criterion

layer8• 2 years ago

In the “Bicubic: note the artifacts” comparison images, the bicubic version, regardless of the aliasing, is less blurry and has more detail than the “magic kernel” version. I therefore don’t agree that the latter is “visually, far superior”. There is at least some trade-off.

herf• 2 years ago

This uniform b-spline is the same one used often as a "Gaussian" approximation (three box filters) - see Paul Heckbert's 1986 paper here (apparently done at NYIT in the early 1980s with help from Ken Perlin):

https://dl.acm.org/doi/pdf/10.1145/15886.15921

raphlinus• 2 years ago

The page mostly talks about image resampling where the goal is more or less preserving all frequencies, but it's also extremely effective at implementing Gaussian blur. Basically, you do n iterations of sampling by 1/2 using this kernel, followed by a very small FIR filter, then n iterations of upsampling 2x using the same kernel. Here, n is essentially log2 of the blur radius, and the total amount of computation is essentially invariant to that radius. All these computations are efficient on GPU - in particular, the upsampling can be done using vanilla bilinear texture sampling (which is very cheap), just being slightly clever about the fractional coordinates.

It works well because, as stated, the kernel does a good job rejecting frequencies prone to aliasing. So, in particular, you don't get any real quality loss from doing 2x scale changes as opposed to bigger steps (and thus considerably larger FIR support).

I have some Python notebooks with some of these results, haven't gotten around to publishing them yet.

kragen• 2 years ago

I look forward to being able to read your notebooks!

leguminous• 2 years ago

I have done something like this with a Lanczos kernel (a=1) downsizing repeatedly by 2x, a small Gaussian kernel, and then repeatedly upsizing by 2x with simple hardware bilinear sampling.

The (2D) Lanczos downsizing can be done with only four samples using the bilinear sampling tricks that you mention, and I avoided expensive trigonometric functions, divisions, and the singularity at 0 by using an even 8th order polynomial approximation. I would be curious to see the results using this kernel, but the Lanczos is so far the best that I've tried.

The magic (image resampling) kernel

Related Stories