Primary brain tumours make up less than 2% of cancers and statistically occur in around 250,000 people a year globally. Medical resonance imaging (MRI) plays a pivotal role in the diagnosis of brain tumours and advanced imaging techniques can precisely detect brain tumours. On this note, Dr Sudhakar Tummala, Assistant Professor, Department of Electronics and Computer Engineering, has published a paper titled, “Classification of Brain Tumour from Magnetic Resonance Imaging using Vision Transformers Ensembling” in the journal Current Oncology having an impact factor of 3.1. The paper highlights the pioneering breakthrough made in the development of vision transformers (ViT) in enhancing MRI for efficient classification of brain tumours, thus reducing the burden on radiologists.

Abstract of the paper

The automated classification of brain tumours plays an important role in supporting radiologists in decision making. Recently, vision transformer (ViT)-based deep neural network architectures have gained attention in the computer vision research domain owing to the tremendous success of transformer models in natural language processing. Hence, in this study, the ability of an ensemble of standard ViT models for the diagnosis of brain tumours from T1-weighted (T1w) magnetic resonance imaging (MRI) is investigated. Pretrained and fine tuned ViT models (B/16, B/32, L/16, and L/32) on ImageNet were adopted for the classification task. A brain tumour dataset from figshare, consisting of 3064 T1w contrast-enhanced (CE) MRI slices with meningiomas, gliomas, and pituitary tumours, was used for the cross-validation and testing of the ensemble ViT model’s ability to perform a three-class classification task. The best individual model was L/32, with an overall test accuracy of 98.2% at 384 × 384 resolution. The ensemble of all four ViT models demonstrated an overall testing accuracy of 98.7% at the same resolution, outperforming individual model’s ability at both resolutions and their ensemble at 224 × 224 resolution. In conclusion, an ensemble of ViT models could be deployed for the computer-aided diagnosis of brain tumours based on T1w CE MRI, leading to radiologist relief.

A brief summary of the research in layperson’s terms

Brain tumours (BTs) are characterised by the abnormal growth of neural and glial cells. BTs causes several medical conditions, including the loss of sensation, hearing and vision problems, headaches, nausea, and seizures. There exist several types of brain tumours, and the most prevalent cases include meningiomas (originate from the membrane surrounding the brain), which are non-cancerous; gliomas (start from glial cells and the spinal cord); and glioblastomas (grow from the brain), which are cancerous. Sometimes, cancer can spread from other parts of the body, which is called brain metastasis. A pituitary tumour is another type of brain tumour that develops in the pituitary gland in the brain, and this gland primarily regulates other glands in the body. Magnetic resonance imaging (MRI) is a versatile imaging method that enables one to noninvasively visualise inside the body, and is in extensive use in the field of neuroimaging.

There exist several structural MRI protocols to visualise inside the brain, but the prime modalities include T1-weighted (T1w), T2-weighted, and T1w contrast-enhanced (CE) MRI. BTs appear with altered pixel intensity contrasts in structural MRI images compared with neighbouring normal tissues, enabling clinical radiologists to diagnose them. Several previous studies have attempted to automatically classify brain tumours using MRI images, starting with traditional machine learning classifiers, such as support vector machines (SVMs), k-nearest-neighbour (kNN), and Random Forest, from hand-crafted features of MRI slices. With the rise of convolutional neural network (CNN) deep learning model architectures since 2012, in addition to emerging advanced computational resources, such as GPUs and TPUs, during the past decade, several methods have been proposed for the classification of brain tumours based on the finetuning of the existing state-of-the-art CNN models, such as AlexNet, VGG16, ResNets, Inception, DenseNets, and Xception, which had already been found to be successful for various computer vision tasks.

Despite the tremendous success of CNNs, they generally have inductive biases, i.e., the translation equivariance of the local receptive field. Due to these inductive biases, CNN models have issues when learning long-range information; moreover, data augmentation is generally required for CNNs to improve their performance due to their dependency on local pixel variations during learning.Therefore, in this work, the ability of pretrained and fine tuned ViT models, both individually and in an ensemble manner, is evaluated for the classification of meningiomas, gliomas, and pituitary tumours from T1w CE MRI at both 224 × 224 and 384 × 384 image resolutions.

Dr Sudhakar Tummala has mentioned the social implications of the research by expounding that the computer-aided diagnosis of brain tumours from T1w CE MRI using an ensemble of fine tuned ViT models can be an alternative to manual diagnoses, thereby reducing the burden on clinical radiologists. He also explains the future prospects of his research, which is to add explainability to the ensemble model predictions and to develop methods for precise contouring of tumour boundaries.

Details of Collaborations

Prof Seifedine Kadry, Department of Applied Data Science, Noroff University College, Kristiansand, Norway.

Dr Syed Ahmad Chan Bukhari, Division of Computer Science, Mathematics and Science, Collins College of Professional Studies, St. John’s University, New York, USA.