Deep Learning on Image Denosing: An Overview

Abstract

基于深度学习的判别学习对于解决高斯噪声的问题十分游泳，并且对于真实的噪声也有一定的效果

本文分为几部分：白噪声图像；真实噪声图像；盲去噪；包含噪声、模糊和低分辩率的混合图像。分析了不同深度学习方法的动机和准则，与最新的方法作对比，最后指出了潜在的挑战和未来研究方向

Q：白噪声？盲去噪？

1 Introduction

相机捕捉的是已经退化的图像，受到光和噪声的干扰，噪声在传输和潜在压缩中产生

50 年前非线性和非自适应滤波器被用于图像去噪，之后基于稀疏的方法被应用，为了恢复潜在图像的细节，先验知识（总变分正则）可以使得噪声图像变得光滑，虽然很多方法效果不错，但是都有各自的缺点，例如有的需要对测试集进行优化

深度学习的技术发展开来，并逐渐被改进，例如损失函数 MSE 到 entropy 损失，对偶拉格朗日乘子，贪婪算法等被用于神经网络，之后构建一个全新的神经网络结构对于去噪很有意义，不论是增强他的深度或者改变他的激活函数

卷积神经网络被提出，但是面临很多的问题，例如梯度消失，激活函数的计算损失较大，硬件设备的计算速度较慢。2012 年，Alex 网络被提出，在 2015 年，深度神经网络被应用于图像去噪，DnCNN 解决多重 low-level 任务，包含 conv、batchnorm 和 Relu。Resnet 被提出解决去噪、超分辨和 JPEG
去遮挡。为了权衡去噪的表现和速度，CNLNet 被提出来有效的取出彩图噪声

FFDNet 提升了去噪速度并且可以处理盲噪声。GAN 的提出可以处理非配对的噪声图像

2 Fundamental frameworks of deep learning methods for image denoising

2.1 Machine learning methods for image denoising

监督学习：
$$
y = x + \mu
$$
无监督学习：给定训练样本寻找pattern而不是标签匹配

半监督学习：将模型应用于给定的数据分布来学习，然后用来给无标签样本标签

2.2 Neural networks for image denoising

神经网络包括神经元，输入$X$，激活函数$f$，权重$W=[W^{0}, W^{1}…W{n-1}]$和偏置$b=[b^{0}, b^{1}…b{n-1}]$

多层神经网络被称为MLP，下图中包含两层，隐藏层和输出层

process
$$
f(X;W;b) = f(W^{n}f(W{n-1}…f(W^{0}X+b{0})…b^{n-1})+b{n})
$$

2.3 CNNs for image denoising

LeNet—AlexNet—VGGNet—GoogLeNet

VGG

GoogLeNet

Drawbacks：网络很深可能会产生梯度消失或者梯度爆炸；过拟合——>解决 ResNet

ResNet

GAN

Generator and Discriminator

2.4 Hardware and software used in deep learning

Hardware : Nvidia AMD

Software : Caffe Theano Matconvnet TensorFlow Keras PyTorch

3 Deep learning techniques in image denoising

3.1 Deep learning techniques for additive white noisy-image denoising

白噪声图片（AWNIs）

Gaussian，Poisson，Salt，Pepper，Multiplicative noisy images

3.1.1 CNN/NN for AWNI denoising

2017 DnCNN

多尺度底层视觉工作：图像去噪，超分辨，去块滤波

残差学习——深层神经网络——递归和自适应特征提取——改进 CNNs，深度神经网络，残差学习，多尺度知识（计算复杂度较大，内存损耗多）——增强感受野——通过多尺度交叉路径的残差连接压制噪声

改变网络结构框架

从 CNN 的多个输入中融合特征：一个样本的不同部分作为不同网络的多个输入；一个样本作为输入的不同视角；不同通道作为 CNN 的输入
改变损失函数
增强 CNN 的深度或宽度：通过增强网络的深度和宽度来扩大感受野的大小
增加 CNN 的一些辅助插件：例如激活函数，空洞卷积（dilated convolution），全连接层和池化操作来增强 CNN 的表达能力
引入跳跃连接或级联操作：跳跃连接能增强级联操作

3.1.2 CNN/NN and common feature extraction methods for AWNI denoising

Weak edge-information noisy images

进行域与域之间的转变

non-linear noisy images

CNN 提取特征；利用核函数的方法将获取的非线性特征映射到线性；利用残差学习重建潜在清晰图像

high dimensional noisy images

CNN 与降维方法相结合。PCA：卷积操作提取特征，PCA 降维，卷积处理降维的特征重建图像

non-salient noisy images

信号处理可以让 CNN 提取非显著特征，例如，跳跃连接

tasks involving high computational costs

3.1.3 Combination of optimization method and CNN/NN for AWNI denoising

discriminative learning；prior knowledge via regular term of loss function

denosing speed

GAN 估计噪声，处理其他任务，图像恢复和超分辨率

基于 CNN 经验的贪婪算法和迁移学习能加速普遍的算法

denoising performance

基于 CNN 的优化方法使得噪声图片更加光滑，CNN 的全变分去噪减少了噪声像素的有效性。GAN 最近邻算法有效地噪声

3.2 Deep learning techniques for real noisy image denoising

single end-to-end CNN

改变网络结构。多尺度知识对图像去噪很有效。

提取细节特征：recurrent connections

处理未知真实噪声：residual structure was utilized to facilitate low-frequency features

提取通道更多潜在特征：attention

the combination of prior knowledge and CNN

3.4 Deep learning techniques for hybrid noisy image denoising

混合噪声图像：噪声、模糊、低分辨率等

突发技术 burst techniques：suffered from effects of noise and camera shake

4 Experimental results

4.1 Datasets

4.1.1 Training datasets

灰度噪声 gray-noisy

训练高斯去噪和盲去噪：

BSD400 dataset：png：400；size：180*180
Waterloo Exploration Database：png：4744 nature images

颜色噪声 color-noisy

BSD432
Waterloo Exploration Database
polyU-Real-World-Noisy-Images datasets：real noisy images：100；size：2784*1856

4.1.2 Test datasets

灰度噪声 gray-noisy

测试高斯噪声和盲噪声：

Set12：contain 12 scenes
BSD68：contain 68 nature images

颜色噪声 color-noisy

CBSD68
Kodak24：24 color noisy images
McMaster：18 color noisy images
cc：15 real noisy images of different ISO(1600, 3200, 6400)
DND：50 real noisy images(clean images were captured by low-ISO images)
NC12：12 noisy images(didn’t have ground-truth clean images)
SIDD：320 images pairs of noisy and ground-truth images(real noisy images from smart phones)
Nam：11 scenes(saved in JPGE format)

4.2 Experimental resultes

4.2.1 Deep learning techniques for additive white noisy-image denoising

Table10：灰度图附加白噪声

Table11：FSIM

放大图片的局部区域，观察去噪效果

4.2.2 Deep learning techniques for real-noisy image denoising

Table15 16：DRDN has the best performance

Table 17：compressed noisy images. AGAN obtains excellent performance.

Table18：real noisy images of different IOS value, SDNet and BRDNet achieved the best and the second-best denoising performance.

blind denoising: noise which is complex in the real world and not subject to rules

FFDNet and ADNet are superior to other methods in blind denoising

4.2.4 Deep learning techniques for hybrid-noisy-image denoising

deep learning techniques based multi-degradation idea：WarpNet DnCNN and MemNet

5 Discussion

Improving denoising performance kills

Enlarge the receptive field can capture more context information: increasing the depth and width of the networks. However, this results in higher computational costs and more memory consumption——>solve: dilated convolution——>higher performance and efficiency, edge information
extra information(prior knowledge) combined with CNN: loss function
combine local and global information: enhance the memory abilities of the shallow layers on deep layers to better filter the noise——>operation: residual operation and recursive operation
single processing methods: ?
Data augmentation: horizontal flip, vertical flip and color jittering; Using GAN to construct virtual noisy images is also useful for image denoising
Transfer learning, graph and neural architecture search
imporve hardware or camera mechanism

trick:

compress neural networks,
reduce the depth or the width of deep neural networks: reduce complexity of these networks
use small convolutional kernal and group convolution: reduce the number of parameters, accelerating the speed of training
fusion of dimension reduction methods: PCA and CNN

Complex noisy images: step by step processing, for example, two steps: one recover high resolution image by CNN, another uses a novel CNN to filter the noise of high-resolution image

several as yet unsolved problems

deeper denoising networkds require more memory resources
deeper denoising networks is not a stable solution for real noisy image, unpaired noisy image and muliti-degradation tasks
real noisy images are not easily captured
difficult to solve unsupervised denoising tasks
more accurate metrics need to be found for image denoising

6 Conclusion

regular Gaussian noise has achieved great success, while the noise in the real world is complex and irregular

Imporve hardware devices in order to suppress the noise for capturing a high-quality image

noisy images with ground truth

本文由 Yonghui Wang 创作，采用知识共享署名4.0 国际许可协议进行许可
本站文章除注明转载/出处外，均为本站原创或翻译，转载前请务必署名
最后编辑时间为: Dec 19, 2024 12:13 pm

Deep Learning on Image Denoising: An Overview