论文标题
通过现实地对双像素数据进行现实建模来减少散焦模糊
Learning to Reduce Defocus Blur by Realistically Modeling Dual-Pixel Data
论文作者
论文摘要
最近的工作显示了使用现代双像素（DP）传感器上可用的两图视图对数据驱动的Defocus Deblurring的令人印象深刻的结果。在这一研究中，一个重大的挑战是访问DP数据。尽管有许多具有DP传感器的相机，但只有有限的数字可以访问低级DP传感器图像。此外，捕获Defocus DeBlurring的训练数据涉及时必的且乏味的设置，需要调整相机的光圈。一些带有DP传感器（例如智能手机）的摄像机没有可调节的光圈，进一步限制了生成必要的训练数据的能力。我们通过提出一个合成生成现实的DP数据的过程来解决数据捕获瓶颈。我们的合成方法模仿了DP传感器上的光学图像形成，并且可以应用于使用标准计算机软件呈现的虚拟场景。利用这些逼真的合成DP图像，我们引入了一个经常性的卷积网络（RCN）体系结构，可改善过度的结果，适合与DP传感器捕获的单帧和多帧数据（例如，视频）一起使用。最后，我们表明，我们的合成DP数据可用于训练针对视频DeBlurring应用程序的DNN模型，其中访问DP数据仍然具有挑战性。
Recent work has shown impressive results on data-driven defocus deblurring using the two-image views available on modern dual-pixel (DP) sensors. One significant challenge in this line of research is access to DP data. Despite many cameras having DP sensors, only a limited number provide access to the low-level DP sensor images. In addition, capturing training data for defocus deblurring involves a time-consuming and tedious setup requiring the camera's aperture to be adjusted. Some cameras with DP sensors (e.g., smartphones) do not have adjustable apertures, further limiting the ability to produce the necessary training data. We address the data capture bottleneck by proposing a procedure to generate realistic DP data synthetically. Our synthesis approach mimics the optical image formation found on DP sensors and can be applied to virtual scenes rendered with standard computer software. Leveraging these realistic synthetic DP images, we introduce a recurrent convolutional network (RCN) architecture that improves deblurring results and is suitable for use with single-frame and multi-frame data (e.g., video) captured by DP sensors. Finally, we show that our synthetic DP data is useful for training DNN models targeting video deblurring applications where access to DP data remains challenging.
