In the paper of “Deep Image Prior”, they did not claim to introduce a novel method. However, they introduced a fact that CNN architecture holds some prior. Well! before delving deeper into the topic, I’ll discuss few things regarding the problem statement or why we need this.
***There are three standard inverse problems discussed here in this paper :***
Super-resolution
When we have a degraded image and we need to enhance the resolution, that’s super-resolution.
Image Denoising
When we have a noisy image, we need to remove the noise and get the better version of the image.
Image Inpainting
In this case we might not know the values of few pixels and we need to guess the value / get the values.
Now, they have solved these problems using the prior which is already in the structure of the CNN. Before starting the formulation, there are few things we need to discuss :
*What is Prior ?*
When we see the LR, we can guess what this image could be. Well, that’s because our brain already has the prior knowledge. And yes that’s a lotus. But can neural network have prior knowledge? without training? Well, in this paper, they have shown this fact.
Continuing the discussion about ‘prior’: We have two types of prior mainly - Implicit prior which can be learned by training the neural network and Explicit prior which can be got from a condition ….
“Explicit priors are predefined constraints or regularizations applied to the model to ensure the SR outputs adhere to certain desirable properties. These priors are derived from domain knowledge or statistical properties of natural images and can significantly enhance the performance”
However, solving this optimization problem (mentioned explicit prior) directly can lead to solutions that do not look natural or realistic.
But in this paper, they used explicit prior where x is the output of neural network. Well, they are using the structure of the neural network which can be used for a strong prior. They also do not claim any new method which can beat SOTA.
The left side image is an example of the result of super resolution task [Source: Paper itself] where DIP can be seen highly competitive with the existing SOTA models and methods.
In the above figure, if x is a clean image and gone through a degradation process to generate X’, our task will be to get X*. To do that, we usually do the maximize the posterior distribution. Using the Bayesian rule, we can get….
$$ p(x|X') ≈ p(x'|x)p(x) $$
** where, p(x'|x)=likelihood p(x)=prior*
Suppose we have an image x and it has gone through a degradation process. In the degradation process, we have added some noise to each pixel of the image. And when we will analyze the likelihood of this degradation process = the probability of an image to be a corrupted image given a clean image, falls under normal distribution. Here, we need to maximize posterior distribution.