Investigation of Single Image Depth Prediction Under Different Lighting Conditions

Abstract

This article investigates the limitations of single image depth prediction (SIDP) under different lighting conditions. Besides that, it also offers a new approach to obtain the ideal condition for SIDP. To satisfy the data requirement, we exploit a photometric stereo dataset consisting of several images of an object under different light properties. In this work, we used a dataset of ancient Roman coins captured under 54 different lighting conditions to illustrate how the approach is affected by them. This dataset emulates many lighting variances with a different state of shading and reflectance common in the natural environment. The ground truth depth data in the dataset was obtained using the stereo photometric method and used as training data. We investigated the capabilities of three different state-of-the-art methods to reconstruct ancient Roman coins with different lighting scenarios. The first investigation compares the performance of a given network using previously trained data to check cross-domains performance. Second, the model is fine-tuned from pre-trained data and trained using 70% of the ancient Roman coin dataset. Both models are tested on the remaining 30% of the data. As evaluation metrics, root mean square error and visual inspection are used. As a result, the methods show different characteristic results based on the lighting condition of the test data. Overall, they perform better at 51° and 71° angles of light, so-called ideal condition afterward. However, they perform worse at 13° and 32° because of the high density of shadows. They also cannot reach the best performance at 82° caused by the reflection that appears on the image. Based on these findings, we propose a new approach to reduce the shadows and reflections on the image using intrinsic image decomposition to achieve a synthetic ideal condition. Based on the results of synthetic images, this approach can enhance the performance of SIDP. For some state-of-the-art methods, it also achieves better results than previous original RGB images.