repository
Accurate Blur Decomposition From a Single Image Using Conditional GANs
- Author(s)
- 이태복
- Alternative Author(s)
- Tae Bok Lee
- Advisor
- 허용석
- Department
- 일반대학원 인공지능학과
- Publisher
- The Graduate School, Ajou University
- Publication Year
- 2023-08
- Language
- eng
- Alternative Abstract
- Single image blur decomposition, which is also known as image deblurring, is a fundamental task in computer vision with a wide range of practical applications. In this study, we investigate the limitations of existing blur decomposition methods in three specific areas: single-to-single face, single-to-video face, and single-to-video general blur decomposition. We aim to identify the shortcomings in these approaches and propose novel solutions to overcome these limitations. <br> <br>For single-to-single face blur decomposition, previous face deblurring methods have utilized semantic segmentation maps as prior knowledge. Most of these methods generated the segmentation map from a blurred facial image, and restore it using the map in a sequential manner. However, the accuracy of the segmentation affects the restoration performance. Generally, it is difficult to obtain an accurate segmentation map from a blurred image. Instead of sequential methods, we propose an efficient method that learns the flows of facial component restoration without performing segmentation. To this end, we propose a multi-semantic progressive learning (MSPL) framework that progressively restores the entire face image starting from the facial components such as the skin, followed by the hair, and the inner parts (eyes, nose, and mouth). Furthermore, we propose a discriminator that observes the reconstruction-flow of the generator. In addition, we present new test datasets to facilitate the comparison of face deblurring methods. Various experiments demonstrate that the proposed MSPL framework achieves higher performance in facial image deblurring compared to the existing methods, both qualitatively and quantitatively. <br> <br>For single-to-video face blur decomposition, we introduce a novel framework for continuous facial motion deblurring that restores the continuous sharp moment latent in a single motion-blurred face image via a moment control factor. Although a motion-blurred image is the accumulated signal of continuous sharp moments during the exposure time, most existing single image deblurring approaches aim to restore a fixed number of frames using multiple networks and training stages. To address this problem, we propose a continuous facial motion deblurring network based on GAN (CFMD-GAN), which is a novel framework for restoring the continuous moment latent in a single motion-blurred face image with a single network and a single training stage. To stabilize the network training, we train the generator to restore continuous moments in the order determined by our facial motion-based reordering process (FMR) utilizing domain-specific knowledge of the face. Moreover, we propose an auxiliary regressor that helps our generator produce more accurate images by estimating continuous sharp moments. Furthermore, we introduce a control-adaptive (ContAda) block that performs spatially deformable convolution and channel-wise attention as a function of the control factor. Extensive experiments on the 300VW datasets demonstrate that the proposed framework generates a various number of continuous output frames by varying the moment control factor. Compared with the recent single-to-single image deblurring networks trained with the same 300VW training set, the proposed method show the superior performance in restoring the central sharp frame in terms of perceptual metrics, including LPIPS, FID and Arcface identity distance. The proposed method outperforms the existing single-to-video deblurring method for both qualitative and quantitative comparisons. In our experiments on the 300VW test set, the proposed framework reached 33.14 dB and 0.93 for recovery of 7 sharp frames in PSNR and SSIM, respectively. <br> <br>For single-to-video general blur decomposition, while recent studies have proposed methods for extracting latent sharp frames from a single blurred image, they still suffer from limitations on restoring satisfactory images. In addition, most existing methods are limited to decomposing a blurred image into sharp frames with a fixed frame rate. To address these problems, we present a Arbitrary Time Blur Decomposition TripleGAN (ABDGAN) that restores sharp frames with flexible frame rates. Our framework plays a min-max game consisting of a generator, a discriminator and a time-code predictor. The generator serves as a time-conditional deblurring network, while the discriminator and the label predictor provide feedback to generator on producing realistic and sharp image depending on given time code. To provide adequate feedback for the generator, we propose a critic-guided (CG) loss by collaboration of the discriminator and time-code predictor. We also introduce a pairwise order-consistency (POC) loss that imposes a stronger symmetric constraint to improve the restoration accuracy. Our experiments show that our method outperforms previously reported methods in both qualitative and quantitative evaluations.
- Fulltext
- Appears in Collections:
- Graduate School of Ajou University > Department of Artificial Intelligence > 4. Theses(Ph.D)
- Files in This Item:
- There are no files associated with this item.
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.
-
- License
- STATISTICS
- Total Visit :5,040,083
- Total Download :2,100
- Today View :7,823
