Detection of Face Morphing Attacks
Identification tasks with identity cards, passports and visa are often performed automatically by biometric face recognition systems. Criminals can trick these systems, such that two people can use the same passport for authentication. This attack (morphing attack) is performed by fusing two face images to a synthetic face image that contains characteristics of both people. Using this image on a passport, both people are authenticated by a biometric face recognition system.
Illustration of a face morphing attack (left and right: source images, center: morphing attack).
Deep Morphing Detection
To generate a face morph, we align two face images, such that each facial feature (e.g. left corner of the right eye) is at the same position in both images. Since the haircut and the ears are difficult or even impossible to align and usually ignored by commercial facial recognition systems, we blend only the face region and stitch it into one of the aligned input images. The transitions are calculated individually for different frequency bands and an optimal stitch line is estimated.
In order to decide whether a face image is authentic or crated by a morphing algorithm, we trained deep convolutional neural networks on morphed and original face images. Since we focus on semantic image content, like highlights in the eyes or the shape and appearance of facial features to determine the authenticity of a face image, we applied several preprocessing steps on our training data to remove sensor or camera specific information. In addition to a decision about the authenticity of an image, we are also interested in a reason for decision. For this purpose, we analyze the regions in the face image that were relevant for the networks decision using the LRP Toolbox. This might also be useful when involving humans to the decision making. It helps to find suspicious regions in a face image that needs to be checked by a human as well as it helps to identify and classify artifacts that were created during a morphing process so that experts can be trained to identify them.
Results of a Layer-wise Relevance Propagation (LRP) algorithm on face morphs. Booth face morphs were correctly classified as morphs by a neutral network. The regions that were relevant for this decision are highlighted.
Robustness against Image Compression
Face images on the chip of electronic Machine Readable Travel Documents are strongly compressed using JPEG or JP2000 compression methods. We analyzed the effect of these lossy compression methods on the accuracy of different detection methods for morphed face images. We found that our DNN-based detections methods are already very robust against image compression. Our DNN-based methods are still able to detect them, if artifacts, which would be used by a human to classify a face image as a morph, are not recognizable for a human inspector. Further investigations in the intermediate steps of our detectors show that they learned features that are already invariant against compression.
Morphed tip of the nose with three different compressions: no compression, JPEG compression, JP2000 (left to right). While the ghosting artifact at the right nostril is easily recognizable in the uncompressed image, it looks completely different and is hard to see in the compressed images.
Neural Morph Enhancement
One important step in the process of creating a morphed face image is the blending of two aligned images. During this process, high frequency details, like wrinkles, scars or pore structures are smoothed or get lost and the resulting image appears dull. A detector might use such characteristics to detect morphed faces, but a professional attacker can manually enhance such details to stay undetected. We developed a method based on Neural Style Transfer that counters this image quality degeneration caused by blending. Thus, we are able to generate morphed face images with characteristics similar to the genuine input images. Our detectors were not able to detect such improved images in many cases at first. After adding the improved morphs to our pool of training data, our detectors can reliable detect them too. In addition, our detectors become even better in detecting morphs that have been improved by post-processing methods, which are not reflected by the training data. Improving the face morphs for our training data thus helped to improve the robustness of our detectors against unknown post-processing methods and to develop detectors that are more general
Simple morph (left) vs. improved morphed face image (right): The improved image has a higher contrast and appears less dull than the simple morph. In particular, the freckles and the eye appear more sharp and realistic.
Publications
C. Seibold, W. Samek, A. Hilsmann, P. Eisert
Accurate and Robust Neural Networks for Face Morphing Attack Detection, Journal of Information Security and Applications, vol. 53, August 2020. [URL], [arXiv]
C. Seibold, A. Hilsmann, A. Makrushin, C. Kraetzer, T. Neubert, J. Dittmann, P. Eisert
Visual Feature Space Analyses of Face Morphing Detectors to Predict Generalization Ability, IEEE Workshop on Information Forensics and Security (WIFS 2019), Delft, Netherlands, December 2019.
C. Seibold, A. Hilsmann, P. Eisert
Style Your Face Morph and Improve Your Face Morphing Attack Detector, 18th International Conference of the Biometrics Special Interest Group (BIOSIG 2019), Darmstadt, Germany, September 2019. [arXiv]
A. Makrushin, C. Kraetzer, J. Dittmann, C. Seibold, A. Hilsmann, P. Eisert
Dempster-Shafer Theory for Fusing Face Morphing Detectors, Proc. European Signal Processing Conference (EUSIPCO), A Coruna, Spain, September 2019. 10.23919/EUSIPCO.2019.8902533
C. Seibold, W. Samek, A. Hilsmann, P. Eisert
Detection of Face Morphing Attacks by Deep Learning, Proc. 16th Int. Workshop on Digital Forensics and Watermarking (IWDW2017), Magdeburg, Germany, August 2017.