Today’s noise reduction software is capable of incredible results. Images that couldn’t be salvaged in the past can be made quite clean with modern denoise algorithms. But what is the real benefit of these tools compared to capturing more light in the first place?
Today, I’ll answer that question numerically by measuring the performance of some different noise reduction algorithms versus capturing more light. I’m going to focus especially on DxO’s PureRaw 4 software (reviewed here on Photography Life) both because of its popularity and because of its high performance. I’ve also tested more conventional noise reduction algorithms that don’t rely on machine learning.
Modern Noise Reduction Performance
Let’s look at an example of an image with excess noise – a reject photo taken at 20,000 ISO on my Nikon D500:
Whoa, noisy! Above, I’ve indicated the crop I’ll be using to show you what it looks like up close.
Frankly, without any denoising, the result is horrific. I tried denoising it using a non-machine-learning algorithm in Rawtherapee, and also with the machine learning algorithm in DxO PureRaw 4:
I think the results speak for themselves. Traditional noise reduction algorithms don’t perform as well as today’s machine learning algorithms, like those used by DxO, Topaz, and now even Adobe. That said, you still don’t get perfect quality in the processed image because of the high levels of noise in the original.
Noise Reduction, or Capturing More Light?
What I rarely see in such tests is a comparison to capturing more light in the field. How do today’s algorithms compare to simply gathering more light?
In other words, if I could have taken the exact same shot but with twice or even four times longer a shutter speed, how would the best noise reduction algorithms compare? We’ve all heard it said that “ISO 6400 is like ISO 800 now” and various claims like that. Well, I’ve done just such a test by using a sturdy tripod, a cable release, and a test subject of a bill of money:
To really see the effects of the noise reduction algorithms, I have used a tight crop:
I took successive photos of this scene at shutter speeds of 1/800, 1/400, 1/200, 1/100, 1/50, and 1/25 second. This resulted in capturing one additional stop of light each time. Correspondingly, I lowered my ISO each time. Here are the results:
In terms of recovering detail and image quality, where do modern noise reduction algorithms stand in the list? To measure this, we need an objective, mathematical standard of measuring image similarity.
There are many algorithms to measure deviation from an ideal or “ground truth” image. After testing a half-dozen image similarity measures, I found one that was very good at measuring image quality loss due to noise: the so-called UIQ or “Universal Image Quality Index.”
According to Zhou Wang and Alan C. Bovik, who published this algorithm in 2002, it measures a “loss of correlation, luminance distortion, and contrast distortion”, which as I found out, roughly corresponds to the presence and perception of detail.
I used this UIQ algorithm to measure the noise in a variety of images – some with noise reduction applied, some simply taken with more light/a lower ISO in the first place. How many stops are you effectively gaining with today’s best noise reduction? These are the results:
A score of one is a perfect score. The image labeled “original” is the one taken at ISO 6400 and 1/800 second with no noise reduction applied. My ideal image is the one taken at 1/25 second and base ISO 200, which is five stops more light than the original photo. (I’ve labeled this “five stops” in the graphic above, and by definition, it gets a perfect score of 1.)
You can see that in this comparison, there is no doubt – a machine learning noise reduction algorithm like those found in DxO PureRaw 4 are a clear step up over traditional noise reduction algorithms. Such traditional algorithms score similarly to a one-stop improvement, whereas DxO PureRaw 4 is somewhere between one and two stops.
Here’s how this looks in an example image, compared to the photo taken at ISO 1600 (two stops better than the original ISO 6400 shot):
Here, you can see that DxO’s result looks great. There isn’t much obvious noise. However, there also is less detail – the image with two more stops of light clearly has finer details on the parrot’s face. This is why the UIQ index scores the two photos about the same – and if anything, gives the edge to the photo with two more stops of light.
I’d also like to show a comparison against traditional noise reduction, such as the one found in Rawtherapee or Darktable:
The DxO image clearly looks better to me. But something else also caught my eye: the dashed lines on the face of the parrot have been transformed by DxO into contiguous lines! This shows that that machine learning algorithms do invent a little detail via interpolation at a micro level. You can see it very clearly in the comparison below (versus the “ideal” image taken at base ISO and 1/25 second):
This shows that in a way, DxO PureRaw 4 and probably other machine-learning denoising algorithms are less like denoisers and more like “re-drawing algorithms.” They use a network trained on millions of images to decide what details to interpolate. By comparison, the traditional denoising algorithm in the previous comparison did not do the same thing.
Discussion
There is no doubt that DxO PureRaw 4’s DeepPrimeXDs algorithm does an outstanding job. It can give you decent images even if you give it noisy slush taken at ISO 20,000, and some photos today are salvageable that weren’t in the past.
At the same time, such algorithms are not a substitute for getting more light – when you can get more light, that is. I don’t buy into the idea that today’s best noise reduction gets you 3, 4, 5, or even more stops of improvement in high-ISO images. Instead, it offers around a two-stop improvement in performance relative to an unedited photo, and about one stop of improvement relative to traditional noise reduction algorithms.
Moreover, DxO PureRaw 4 can add a small amount of interpolation on a fine scale, effectively guessing extremely fine detail in order to achieve results – which is something not everyone is comfortable with, including myself.
Finally, machine-learning denoising makes the most difference in the ISO 6400+ range. Modern sensors do very well at ISO 3200 and below, and noise in such images can be cleaned with a traditional algorithm without major issues. And, in my experience, I find the best images to be taken at these lower ISO values anyway, because the stronger light gives better color and detail.
Therefore, while DxO PureRaw 4 and other machine learning noise reduction can certainly improve noisy images better than traditional algorithms, it still pays to optimize your camera settings if you want the best image quality. It’s better to capture more light than to use software to make up for excessively high ISOs. And no software can make a high-ISO photo look like it was taken at base ISO.
Note: I’d like to thank DxO for providing me with a license to use this software for testing purposes.
