Ah, this is a great idea, and I can see how this could be extended beyond simple deblocking.
In essence, it computes the uncertainty from the JPEG quantization step (the source of most of JPEG’s compression) and uses that information to select a decoding that minimizes a particular error metric (here, discontinuities at block boundaries).
But it’s not limited to just that, in principle. It could optimize towards any decoded block within the quantization range given a suitable prior - and I imagine that this could be paired with e.g. a DNN to faithfully reconstruct an image from a JPEG with much better accuracy.
Additionally, I'd love to see a decoder add noise proportional to the uncertainty and surrounding structure as over smoothed images look unnatural as well. See activity masking demo https://people.xiph.org/~jm/daala/pvq_demo/
yeah, the deblocked image looks a little softer. Makes sense, the optimization objective encourages it. Sometimes the block boundary aligns with an image boundary. Agree that incorporating a DNN could be a cool solution.
Test image still has perceptible blocking. Description suggests it should be possible to reconstruct source values/smoothness staying in stored intervals, and yet there are still visible boundaries.
Lower right corner you can clearly trace 4 horizontal blocks, looks like algorithm tries hard to maintain encoded hard boundaries despite there being gradual color shift. Maybe it needs to look further than direct neighbor block for clues.
Unacceptable. Even though swiss-german doesn't have any spelling rules, I hereby declare a public vote to deport google from switzerland, because it's clearly spelled "Chnuschperli" in proper swiss-german.
Edit: Sorry, I forgot HN is allergic to jokes, obviously this is turning into reddit. I apologize.
So what does Knusperli even mean? It seems to mean “crunchy” or something from a Google search but Translate turns up nothing. Is the meaning at all related to the project, or is it just a cute name?
Yes, it means "crunchy thingy" basically. The name doesn't have any relation to the project. It's small pieces of fish fried in a dough mantle.
I was jokingly complaining about the spelling because the previous names "Brötli" and "Zöpfli" of similar projects were 'properly' spelled. They don't have anything to do with the name either. "Brötli" and "Zöpfli" are basically types of bread.
While Knusperli is a very german way of spelling a swiss german word. Though, it is what you would find in a supermarket.
> No need to 'joke' about deportation. That's what disgusted me.
Yeah, I should have considered global cultural influences more before posting. Sarcastically joking about deportation is probably a bit too swiss for an international audience to recognize it as such. Let me assure your though, this was an attempt by a swiss person desperately trying to be funny.
I think visually speaking, while the deblocked version is smoother overall, the blockiness added extra perception of noise/texture. The extra noise almost always increase the perception of sharpness and level of details, and you can see from the braids coming out of the hat that it does look sharper with the blocking version, although the wall is smoother. Would be great to include more sample images in the page.
x264 video codec developer Dark Shikari blogged about [1][2] this "fake sharpness" phenomenon. Early H.264 encoders produced excessively smooth, deblocked video, which made them look worse compared to MPEG-4 ASP (like Xvid), despite former being a newer, objectively better format. Wavelet-transform compressors like JPEG2000 had a similar problem vs. blocky JPEGs.
People's tastes are subjective, though, so it's good to have options.
I would assume that JPEG like most lossy compression algorithms is specifically defined in terms of the correct uncompressed output for a given compressed input. This means a "deblocking JPEG decoder" is effectively actually just a bad decoder that gives the wrong output (albeit for some outputs maybe you happen to prefer this).
The definitions are this way around because it's the only way to have an opportunity to improve encoding quality over time, if you start by defining how encoding must be done then you lock in bad ideas. Lame for example is far, far better at encoding MPEG audio layer III than the demo encoders shipped when the standard was written, and this is only possible because the standard says how to turn MP3 back into PCM data, not how to encode PCM data as MP3.
Agreed, I believe the image of Lenna should be discouraged, as some people might find the image offending and unwelcoming to women (it originally appeared in a 1972 Playboy magazine issue).
Considering Lena doesn't object and by all accounts is quite happy her image is used, there really is no reason not to use her image. If you real want you can use Fabio instead.
Dissenting opinion: I think the blocky version looks better when not zoomed in. The sample on the left is less smoothed, and doesn't look as airbrushed. It looks better when zoomed in, but not when are regular zoom.
This is one of those nice "let's do this properly" projects google does. I'd put into the same category as their work on compression. Unsexy and "so last decade" at first glance, but decidedly useful and universally applicable, given the number of jpg files created since the dawn of time.
That being said, I'd really wish people would stop using lossy formats for everything. I can live with it for simple icons and such (although, why not svg?) but photos? come on!
Of course, that would have to come with a nice connection for everyone.
Oh and this is definitely going to make for some head scratching for devs who assume image decoding is somehow deterministic. :D
Like that time they stole Jarek Duda ANS and are still trying to patent it fraudulently claiming to have invented it 4 years after Jareks public domain publication?
I'm not sure if you are joking, but it does seem that more interoperability is usually a good thing.. (especially for something like a jpeg decoder - maybe it's less useful to have a toaster interface smoothly with some coffee beans)
Assuming interoperability to be a good thing, then, coding something like a c# translation of this is a stimulating and useful intellectual exercise relative to like reading a book or solving a prescribed math problem. Assuming coding problems aren't like fossil fuel for jobs or something, and society hasn't yet reached the state where we can just neglect technology and say 'good enough,' given that (at least according to people I see while locomoting around town), there are still diseases, disabilities (physical and psychological), aging, puns, death, overpopulation, homelessness, variety of experience (I don't know what the correct word for this is, but something like 'non-crunchy-multitude-of-choice'), etc. all which may have solutions with sufficiently advanced technology (with the caveat that it may be like a pandora's box).
(You could probably superficially post-justify a solution to these kind of problems in society by strapping an RC radio to everyones head and saying 'karma', but I would guess you can't do that without the being enacting this breaking large parts of the social contract, and essentially ridding life of substance, e.g. 'this thing happened to you because you ate a fish when you were a child, and that fish had a family, and so, relatively, your brain is like a fish-size (lets say chicken because birds are actually pretty smart in the animal hierarchy) to this AI we've constructed, and thus it's eating you, and so therefore, its karmically balanced (that's not really a solution is it, I mean it is, but it's like a military tit for tat solution (even if done subtly) rather than a diplomatic / medical type solution, which would provide more leeway for things like basic fight-or-flight response of humans / miscommunication / speed of actual human thought / etc., which doesn't really fit in the tit-for-tat model)' ). Wow I wrote freaking thesis here (and I didn't really put that long of thought into it, so judge as such, like git commit 1 of an more revised thought).
In essence, it computes the uncertainty from the JPEG quantization step (the source of most of JPEG’s compression) and uses that information to select a decoding that minimizes a particular error metric (here, discontinuities at block boundaries).
But it’s not limited to just that, in principle. It could optimize towards any decoded block within the quantization range given a suitable prior - and I imagine that this could be paired with e.g. a DNN to faithfully reconstruct an image from a JPEG with much better accuracy.