Learning a Convolutional Neural Network with Additional Information

Abstract

Learning representations for object recognition and image quality enhancement with the deep convolutional neural network approach has received a great deal of attention in the past several years and recently has gained widespread popularity in the field of computer vision and image processing. Many convolutional neural networks based researches has shown successful results in image processing and computer vision area by feeding only raw images into the convolutional neural network model to learn network parameters. In supervised learning, every input image in our traingset is "a question" and corresponding label or groundtruth is "a correct answer" that we would have quite liked the algorithms have predicted on that image. Even though the CNN models can be skillful enough to solve a problem by learning a large trainingset, we expect that if CNNs were feed with additional information, as well as with images, that would leads learning in a better way, it may perform better. It is like giving helpful hints when teaching a child.

This thesis presents fingerprint liveness detection and space-varying deblur methods based on CNN. The first fingerprint liveness detection belongs to a classification problem and the second space-varying deblur belongs to a prediction problem of original sharpen image. Instead of training CNNs with input images only, we provide additional information with which CNNs can learn features in a domain specific, or a problem specific way. Simple additional information that can obtain fairly easily but crucial for a given task is used as an additional input of each CNN. Sweat pore map is used as additional information for fingerprint liveness detection and spatial pixel indices are used as additional information for space-varying deblur. For fingerprint liveness detection, the sweat pore map provides enough hint to allow CNN to learn features for specific regions such as regions right at the pores, regions around the pores, regions in the ridges that do not contain pores. Our fingerprint liveness detection method outperforms the best algorithms of fingerprint liveness detection competition 2013(LivDet2013). For CNN based space-varying debur, the spatial pixel indices provide enough hint to allow CNN to learn filters that have stronger response at specific areas of an image. Our non-stationary lens blur method is the first CNN model that directly outputs the restored image from a blurry image, without any assumption or approximation of block-wise spatially invariant blur. The proposed deblurring method provides the state-of-the-art performances. We established procedures for building training sets from real-world lenses and cameras for restoration of lens blur.