VISN: virus instance segmentation network for TEM images using deep attention transformer

Abstract

Abstract The identification of viruses from negative staining transmission electron microscopy (TEM) images has mainly depended on experienced experts. Recent advances in artificial intelligence have enabled virus recognition using deep learning techniques. However, most of the existing methods only perform virus classification or semantic segmentation, and few studies have addressed the challenge of virus instance segmentation in TEM images. In this paper, we focus on the instance segmentation of severe acute respiratory syndrome coronavirus type 2 (SARS-CoV-2) and other respiratory viruses and provide experts with more effective information about viruses. We propose an effective virus instance segmentation network based on the You Only Look At CoefficienTs backbone, which integrates the Swin Transformer, dense connections and the coordinate-spatial attention mechanism, to identify SARS-CoV-2, H1N1 influenza virus, respiratory syncytial virus, Herpes simplex virus-1, Human adenovirus type 5 and Vaccinia virus. We also provide a public TEM virus dataset and conduct extensive comparative experiments. Our method achieves a mean average precision score of 83.8 and F1 score of 0.920, outperforming other state-of-the-art instance segmentation algorithms. The proposed automated method provides virologists with an effective approach for recognizing and identifying SARS-CoV-2 and assisting in the diagnosis of viruses. Our dataset and code are accessible at https://github.com/xiaochiHNU/Virus-Instance-Segmentation-Transformer-Network.