RFTNet: Region–Attention Fusion Network Combined with Dual-Branch Vision Transformer for Multimodal Brain Tumor Image Segmentation
-
Published:2023-12-23
Issue:1
Volume:13
Page:77
-
ISSN:2079-9292
-
Container-title:Electronics
-
language:en
-
Short-container-title:Electronics
Author:
Jiao Chunxia1, Yang Tiejun234, Yan Yanghui1, Yang Aolin1
Affiliation:
1. School of Information Science and Engineering, Henan University of Technology, Zhengzhou 450001, China 2. School of Artificial Intelligence and Big Data, Henan University of Technology, Zhengzhou 450001, China 3. Key Laboratory of Grain Information Processing and Control (HAUT), Ministry of Education, Zhengzhou 450001, China 4. Henan Key Laboratory of Grain Photoelectric Detection and Control (HAUT), Zhengzhou 450001, China
Abstract
Brain tumor image segmentation plays a significant auxiliary role in clinical diagnosis. Recently, deep learning has been introduced into multimodal segmentation tasks, which construct various Convolutional Neural Network (CNN) structures to achieve excellent performance. However, most CNN-based segmentation methods have poor capability for global feature extraction. Transformer is good at modeling long-distance dependencies, but it can cause local information loss and usually has a high computational complexity. In addition, it is difficult to fully exploit the brain tumor features of different modalities. To address these issues, in this paper, we propose a region–attention fusion (RAF) network that combines a dual-branch vision Transformer (DVT), called RFTNet. In RFTNet, the DVT is exploited to capture the delicate local information and global semantics separately by two branches. Meanwhile, a novel RAF is employed to effectively fuse the images of the different modalities. Finally, we design a new hybrid loss function, called region-mixed loss function (RML) to calculate the importance of each pixel and solve the problem of class imbalance. The experiments on BrasTS2018 and BraTS2020 datasets show that our method obtains a higher segmentation accuracy than other models. Furthermore, ablation experiments prove the effectiveness of each key component in RFTNet.
Funder
National Natural Science Foundation of China key specialized research and development program of Henan Province Open Fund Project of Key Laboratory of Grain Information Processing & Control Innovative Funds Plan of Henan University of Technology
Subject
Electrical and Electronic Engineering,Computer Networks and Communications,Hardware and Architecture,Signal Processing,Control and Systems Engineering
Reference49 articles.
1. Işın, A., Direkoğlu, C., and Şah, M. (2016, January 29–30). Review of MRI-based brain tumor image segmentation using deep learning methods. Proceedings of the 12th International Conference on Application of Fuzzy Systems and Soft Computing, ICAFS 2016, Vienna, Austria. 2. A review on brain tumor segmentation of MRI images;Wadhwa;Magn. Reson. Imaging,2019 3. Menze, B.H., Van Leemput, K., Lashkari, D., Weber, M.-A., Ayache, N., and Golland, P. (2010). Medical Image Computing and Computer-Assisted Intervention–MICCAI 2010, Proceedings of the 13th International Conference, Beijing, China, 20–24 September 2010, Springer. 4. A review: Deep learning for medical image segmentation using multi-modality fusion;Zhou;Array,2019 5. Bauer, S., Wiest, R., Nolte, L.-P., and Reyes, M. (2013). A survey of MRI-based medical image analysis for brain tumor studies. Phys. Med. Biol, 58.
|
|