Super Resolution from a Single Image (2009)

http://www.wisdom.weizmann.ac.il/~vision/SingleImageSR.html

by bsilvereagle • 11 years ago

146 42 11 years ago

wisdom.weizmann.ac.il

dang• 11 years ago

Discussed in 2012: https://news.ycombinator.com/item?id=4241266.

radarsat1• 11 years ago

So, if I understand from the abstract, for every pixel they look through the image for similar "pixel neighbourhoods" of the same or varying size, and collect them into a kind of database of examples to be used for scaling up that pixel. Pretty cool idea!

CephalopodMD• 11 years ago

Zoom and enhance

jewbear48• 11 years ago

Is there an implementation of this available?

0942v8653• 11 years ago

There is a link in the old thread: https://news.ycombinator.com/item?id=4242002

CamperBob2• 11 years ago

I guess I fail the IQ test. Where's the download link?

turshija• 11 years ago

I just can't understand how it manages to make this image http://www.pics.rs/i/2cuoM look like this image http://www.pics.rs/i/IEz0N, especially the last line of letters.

vog2• 11 years ago

I guess the key reason why this works so well is because the lower resolution has been calculated from the higher resolution. So you have detailed information on how exactly the downscaling was calculated, which enables you to "cheat".

If you apply the same method to a completely different downscaling method, it probably wouldn't work out as nicely.

The ultimate test would be two different photographs taken at the same time - one with high and one with low resolution. Almost certainly this wouldn't work out as nicely. There is a huge difference between (a) having a low-res image due to physical effects, and (b) having a low-res image calculated by a known algorithm where all parameters are also known.

jessriedel• 11 years ago

Very true, but in principle it's possible to completely characterize the downscaling process effected by a physical camera.

beagle33• 11 years ago

This is plain wrong. As others have pointed out, it is independent of the scaling method - it just looks for similar patches (at different sizes, mirror images, and perhaps even rotations - I don't remember) on the same image to find the best one that matches, and uses that for information.

In this specific case, you have all the letters there to match against, so there's no surprise.

Note that it uses the entire image to look for matches. If you tried to enlarge only the last line, it would probably NOT look as good.

There's a good reason that it works: Many pictures include similar elements at different scales, which lets you infer things from one scale to the other. In fact, in the eighties there was a lot of hype about "fractal compression" based on the same principle, see e.g. http://www.cs.northwestern.edu/~agupta/_projects/image_proce... ; In the end, they couldn't improve on JPEG, and has been essentially forgotten - but ... it did match the JPEG coders at the time in terms of compression rate (did much worse on speed and memory requirements); and, it could decompress pictures to much larger geometry than the original while still looking good -- technically, very similar to what is described in this paper. Everything old is new again.

ricardobeat• 11 years ago

As far as I understood, the letters at the bottom row are rebuilt from patterns found in other parts of the image - a few of them are false positives, see the original chart https://joyerickson.files.wordpress.com/2010/07/eye-chart.jp...

irixusr2• 11 years ago

I think the idea is that an image has a certain amount of self similarity. I.e. skin looks like skin. So the algorithm learns how skin is supposed to look like by looking at all the skin. So every wrinkle is improved by looking at every other wrinkle as opposed to just the pixles in an immediate vicinity.

Now replace "wrinkle" for "patch" and sprinkle some crazy stats and machine learning to get the idea to work.

crazy2be• 11 years ago

There is some discussion of this from 2012, when this was last posted. It looks to be guessing the letters (sometimes incorrectly) based on the larger ones present in the image: https://news.ycombinator.com/item?id=4241978

JTxt• 11 years ago

Thanks, I was about to dig that up. Neat to see this again. (I should check out their more recent research now.)

lambdapie• 11 years ago

The general principal seems to be replacing being distance X from the truth with probability 1, with being distance X/10 from the truth with probability 0.99 and distance X*10 from the truth with probability 0.01.

This is consistent with another reply which states that it guesses the true letter (sometimes incorrectly)

colanderman• 11 years ago

I wonder if the same techniques could be used for compression as well.

irixusr2• 11 years ago

If you mean lossy, why not? Netflix could use this by reducing the information they send out to their users who recreate a lot of what was lost.

I don't think for loss-less though, because they are inferring what a pixel should to be, i.e. they cannot be 100% any bit is every set right, just the probability that it is.

foota• 11 years ago

I believe this is already somewhat similar to say zip compression, which can reference other portions of the compressed file.

Houshalter• 11 years ago

Absolutely. For lossless compression you just need to encode the difference between the predicted pixels and the actual pixels. Which hopefully has a lot of zeros, making it easier to compress. For lossy compression, you just throw away some or all of the difference info.

The biggest issue is speed. This doesn't look very fast to encode or decode. That's the biggest issue with new image compression algorithms, and makes it nonviable for most purposes.

nitrogen• 11 years ago

Some photocopiers have actually replaced numbers with different numbers in copied documents because of pattern-based compression: https://news.ycombinator.com/item?id=6156238

ansgri2• 11 years ago

Photocopiers use Mixed Raster Content (MRC) with JBIG compression, which is indeed able to confuse symbols at the same scale, but across-the-scales substitution is beyond the current MRC capabilities afaik.

beagle33• 11 years ago

Yes, and this was all the rage in the late '80s and early '90s. Everyone was sure this will replace JPEGs. See http://www.cs.northwestern.edu/~agupta/_projects/image_proce... for a description.

Spoiler: It didn't happen. Fractal compression mostly matched JPEG on rate/quality ; it didn't improve on it, though; and it was horrible in both memory use, run time, and memory requirements compared to JPEG. So JPEG it (still) is.

Another Spoiler: If file size was the criterion, BPG would be replacing JPEG these days. That isn't happening either.

happyindeed• 11 years ago

It has a visual style that looks like a "photorealism" painting. Quite amazing. https://en.wikipedia.org/wiki/Photorealism

coldtea• 11 years ago

Hmmm? The idea of photorealism is that it should look as much as an actual photograph (or reality) as possible. And the best such painters, do.

agumonkey• 11 years ago

Interesting to see what some of them did since https://sites.google.com/site/dglasner/

Pretty original research.

crimsonalucard• 11 years ago

This is the algorithm they use in all those movies. "Enhance!"

zurn• 11 years ago

Original code @ https://github.com/stefanv/supreme (repost from downthread). Looks like it's BSD licensed and in Python.

ansgri2• 11 years ago

As I see from http://mentat.za.net/supreme/, this is an implementation of wholly another method, neither single-image nor self-similarity–based.

osdf4• 11 years ago

A lot of things happened since then. References up to spring 2014 are e.g. in our own work to super-resolve arbitrary sized images with convolutional en/decoders: Super-Resolution with Fast Approximate Convolutional Sparse Coding, http://brml.org/uploads/tx_sibibtex/281.pdf (which still has a lot of possibilities to be extended (e.g. color) and improved upon).

nkoren• 11 years ago

Interesting paper; I'd be curious to see examples of how that performs. Are any online?

osdf4• 11 years ago

No, sorry :( Clearly a TODO for some future work.

tmaly• 11 years ago

To clarify my understanding of this post, would it be possible using this method to: 1. start with a high resolution image 2. create a low resolution version 3. using the low resolution version produce a high resolution version that looks good

Super Resolution from a Single Image (2009)

Related Stories