The 3D-aware image synthesis of prohibited items in the X-ray security inspection by stylized generative radiance fields

Abstract

<abstract><p>The merging of neural radiance fields with generative adversarial networks (GANs) can synthesize novel views of objects from latent code (noise). However, the challenge for generative neural radiance fields (NERFs) is that a single multiple layer perceptron (MLP) network represents a scene or object, and the shape and appearance of the generated object are unpredictable, owing to the randomness of latent code. In this paper, we propose a stylized generative radiance field (SGRF) to produce 3D-aware images with explicit control. To achieve this goal, we manipulated the input and output of the MLP in the model to entangle and disentangle label codes into/from the latent code, and incorporated an extra discriminator to differentiate between the class and color mode of the generated object. Based on the labels provided, the model could generate images of prohibited items varying in class, pose, scale, and color mode, thereby significantly increasing the quantity and diversity of images in the dataset. Through a systematic analysis of the results, the method was demonstrated to be effective in improving the detection performance of deep learning algorithms during security screening.</p></abstract>