英语原文共 8 页，剩余内容已隐藏，支付完成后下载完整资料
Learning Raw Image Denoising with Bayer Pattern Unification and Bayer Preserving Augmentation
Jiaming Liu Chi-Hao Wu Yuzhi Wang Qin Xu Yuqian Zhou Haibin Huang
Chuan Wang Shaofan Cai Yifan Ding Haoqiang Fan Jue Wang
Abstract
In this paper, we present new data pre-processing and augmentation techniques for DNN-based raw image denoising. Compared with traditional RGB image denoising, performing this task on direct camera sensor readings presents new challenges such as how to effectively handle various Bayer patterns from different data sources, and subsequently how to perform valid data augmentation with raw images. To address the first problem, we propose a Bayer pattern unification (BayerUnify) method to unify different Bayer patterns. This allows us to fully utilize a heterogeneous dataset to train a single denoising model instead of training one model for each pattern. Furthermore, while it is essential to augment the dataset to improve model generalization and performance, we discovered that it is error-prone to modify raw images by adapting augmentation methods designed for RGB images. Towards this end, we present a Bayer preserving augmentation (BayerAug) method as an effective approach for raw image augmentation. Combining these data processing technqiues with a modified U-Net, our method achieves a PSNR of 52.11 and a SSIM of 0.9969 in NTIRE 2019 Real Image Denoising Challenge, demonstrating the state-of-the-art performance. Our code is available at https://github. com/Jiaming-Liu/BayerUnifyAug.
1.Introduction
Image denoising is one of the fundamental problems in image processing and computer vision, and restoring high quality images from extremely noisy ones remains to be challenging. This can be even worse when it comes to images taken from mobile devices. Due to the use of relatively low-cost sensors and lenses, images captured by mobile cameras can be severely corrupted by high level noise, especially in low-light scenarios. Many denoising methods have been proposed to address this problem, including traditional methods such as NLM [6] and BM3D [10] as well as more recent deep neural network (DNN) based denoising models [26, 9, 31, 19, 28, 23, 27], but their performances are still far from satisfactory on mobile devices.
Recently, thanks to the public raw image denoising datasets [1, 8, 4], denoising raw image data has received more and more attentions and has shown promising results [8, 17, 12]. Raw images are direct readings from images sensors, with camera filter arrays (CFAs) arranged in specific patterns such as the Bayer pattern [5]. These digital signals are further post-processed to obtain RGB images through a complex pipeline including lens shading correction, white balancing, demosaicking, gamma correction, etc. [14]. Therefore, original noise properties that exist in raw images are often distorted in RGB images, making the noise harder to remove afterwards. This means that there are potentially better denoising methods that can be developed on the raw image data [8], compared with many works done in RGB domain. In this work, we study the problem of raw image denoising using DNN, and our focus is on how to train an effective raw image denoising model by proper data pre-processing and data augmentation.
First of all, to perform raw image denoising with DNN models, it is a common practice to pack a Bayer raw image into a 4-channel RGGB image, and feed it into neural networks [8]. With data collected from cameras with different Bayer patterns, a simple solution is to train one model for each pattern. However, this decreases the size of the effective training set and thereby hurts the performance. To fully utilize all training data to achieve better performance, we propose a Bayer pattern unification (BayerUnify) technique to eliminate the differences among Bayer patterns. As illustrated in Fig. 1 (a), flipping and cropping operations are employed to turn a specific CFA pattern into another one, with which we can unify all training images into the same pattern. As a result, all the training data can be used together to optimize a single model to achieve the best possible result.
Data augmentation is a common approach in deep learning to improve model performance by increasing the diversity of a training dataset. However, data augmentation of raw images is not as straightforward as that of RGB images. An example in shown in Fig. 1 (b). Simply flipping the packed 4-channel raw images is erroneous because it results in an image that is impossible in real world. This phenomenon can also be found in other types of augmentation operations such as cropping, transposition, etc. To tackle this problem, we introduce a Bayer preserving augmentation (BayerAug) technique that allows proper augmentation for raw images. As shown in Fig. 1 (b), extra operations are required to correctly flip a raw image.
Figure 1. Demonstration of our proposed (a) Bayer pattern unification and (b) Bayer preserving augmentation. Our method unifies and augments the Bayer raw images without affecting the content, while improper pre-processing or augmentation would disturb the spatial relationship of the raw images and therefore result in artifacts.
Both BayerUnify and BayerAug techniques are simple, yet effective ways for increasing the training data size and diversity for raw image denoising. We apply these techniques to train models
剩余内容已隐藏，支付完成后下载完整资料
资料编号：[254006]，资料为PDF文档或Word文档，PDF文档可免费转换为Word
以上是毕业论文外文翻译，课题毕业论文、任务书、文献综述、开题报告、程序设计、图纸设计等资料可联系客服协助查找。
