Early Bruise Detection in Apple Based on an Improved Faster RCNN Model

Abstract

Bruising is a common occurrence in apples that can lead to gradual fruit decay and substantial economic losses. Due to the lack of visible external features, the detection of early-stage bruising (occurring within 0.5 h) is difficult. Moreover, the identification of stems and calyxes is also important. Here, we studied the use of the short-wave infrared (SWIR) camera and the Faster RCNN model to enable the identification of bruises on apples. To evaluate the effectiveness of early bruise detection by SWIR bands compared to the visible/near-infrared (Vis/NIR) bands, a hybrid dataset with images from two cameras with different bands was used for validation. To improve the accuracy of the model in detecting apple bruises, calyxes, and stems, several improvements are implemented. Firstly, the Feature Pyramid Network (FPN) structure was integrated into the ResNet50 feature extraction network. Additionally, the Normalization-based Attention Module (NAM) was incorporated into the residual network, serving to bolster the attention of model towards detection targets while effectively mitigating the impact of irrelevant features. To reduce false positives and negatives, the Intersection over Union (IoU) metric was replaced with the Complete-IoU (CIoU). Comparison of the detection performance of the Faster RCNN model, YOLOv4P model, YOLOv5s model, and the improved Faster RCNN model, showed that the improved model had the best evaluation indicators. It achieved a mean Average Precision (mAP) of 97.4% and F1 score of 0.87. The results of research indicate that it is possible to accurately and effectively identify early bruises, calyxes, and stems on apples using SWIR cameras and deep learning models. This provides new ideas for real-time online sorting of apples for the presence of bruises.