Publications

Publications

For a complete list of publications, please refer to Google Scholar.

2021

Avinash Kori, Mahendra Khened, Haran Rajkumar, Ganapathy Krishnamurthi. A generalized deep learning framework for whole-slide image segmentation and analysis. Nature scientific reports, 2021.
(.pdf) (abstract)

Histopathology tissue analysis is considered the gold standard in cancer diagnosis and prognosis. Whole-slide imaging (WSI), i.e., the scanning and digitization of entire histology slides, are now being adopted across the world in pathology labs. Trained histopathologists can provide an accurate diagnosis of biopsy specimens based on WSI data. Given the dimensionality of WSIs and the increase in the number of potential cancer cases, analyzing these images is a time-consuming process. Automated segmentation of tumorous tissue helps in elevating the precision, speed, and reproducibility of research. In the recent past, deep learning-based techniques have provided state-of-the-art results in a wide variety of image analysis tasks, including the analysis of digitized slides. However, deep learning-based solutions pose many technical challenges, including the large size of WSI data, heterogeneity in images, and complexity of features. In this study, we propose a generalized deep learning-based framework for histopathology tissue analysis to address these challenges. Our framework is, in essence, a sequence of individual techniques in the preprocessing-training-inference pipeline which, in conjunction, improve the efficiency and the generalizability of the analysis. The combination of techniques we have introduced includes an ensemble segmentation model, division of the WSI into smaller overlapping patches while addressing class imbalances, efficient techniques for inference, and an efficient, patch-based uncertainty estimation framework. Our ensemble consists of DenseNet-121, Inception-ResNet-V2, and DeeplabV3Plus, where all the networks were trained end to end for every task. We demonstrate the efficacy and improved generalizability of our framework by evaluating it on a variety of histopathology tasks including breast cancer metastases (CAMELYON), colon cancer (DigestPath), and liver cancer (PAIP). Our proposed framework has state-of-the-art performance across all these tasks and is ranked within the top 5 currently for the challenges based on these datasets. The entire framework along with the trained models and the related documentation are made freely available at GitHub and PyPi. Our framework is expected to aid histopathologists in accurate and efficient initial diagnosis. Moreover, the estimated uncertainty maps will help clinicians to make informed decisions and further treatment planning or analysis.

Avinash Kori, Parth Natekar, Balaji Srinivasan, Ganapathy Krishnamurthi. Interpreting Deep Neural Networks for Medical Imaging using Concept Graphs. AAAIw, 2021.
(.pdf) (abstract)

The black-box nature of deep learning models prevents them from being completely trusted in domains like biomedicine. Most explainability tech- niques do not capture the concept-based reasoning that human beings follow. In this work, we attempt to understand the behavior of trained models that perform image processing tasks in the medical domain by building a graphical representa- tion of the concepts they learn. Extracting such a graphical representation of the model’s behavior on an abstract, higher conceptual level would help us to unravel the steps taken by the model for predictions. We show the application of our pro- posed implementation on two biomedical problems - brain tumor segmentation and fundus image classification. We provide an alternative graphical representa- tion of the model by formulating a concept level graph as discussed above, and find active inference trails in the model. We work with radiologists and ophthal- mologists to understand the obtained inference trails from a medical perspective and show that medically relevant concept trails are obtained which highlight the hierarchy of the decision-making process followed by the model. Our framework is available at https://github.com/koriavinash1/BioExp.

2020

Avinash Kori. Dynamic Behavior of Coupled Neuron System from a Game-Theoretic Standpoint. NBCT (under review), 2020.
(.pdf) (abstract)

This paper is concerned with the theoretical investigation of game theory concepts in analyzing the behavior of dynamically coupled oscillators. Here, we claim that the coupling strength in any neuronal oscillators can be modeled as a game. We formulate the game to describe the effect of pure-strategy Nash equilibrium on two neuron systems of Hopf-oscillator and later demonstrate the application of the same assumptions and methods to N × N neuronal sheet. We also demonstrate the effect of the proposed method on MNIST data to show the equilibrium behavior of neurons in a N × N neuronal grid for all different digits. A significant outcome of the paper is a modified Hebbian algorithm, which adapts the coupling weights to neural potential resulting in a stable phase difference. Which in turn, makes it possible for an individual neuron to encode input information.

Tahsin Kurc, et.al. Segmentation and Classification in Digital Pathology for Glioma Research: Challenges and Deep Learning Approaches. Frontier in Neuroscience, 2020.
(abstract)

Biomedical imaging is an important source of information in cancer research. Characterizations of cancer morphology and function at onset, progression and in response to treatment provide complementary information to that gleaned from genomics and clinical data. Accurate extraction and classification of both visual and latent imaging features has been an increasingly complex challenge due to the increased complexity and resolution of imaging data. In this paper, we present image analysis methods that employ deep learning approaches to i) segment nuclei in whole slide images of glioma and ii) classify brain tumor cases as either Oligodendroglioma or Astrocytoma using radiographic and histologic imaging data. These methods were submitted to the Computational Precision Medicine satellite event at the 21st International Medical Image Computing and Computer Assisted Intervention (MICCAI) 2018 conference and achieved top scores; 0.87 Dice Similarity Coefficient in segmentation and 0.75-0.90 Accuracy in classification. The results indicate that carefully constructed deep learning networks are able to produce high accuracy in biomedical image analysis and the combination of radiographic with histologic image information improves classification performance.

Avinash Kori, Mahendra Khened, Haran Rajkumar, Balaji Srinivasan, Ganapathy Krishnamurthi. A Generalized Deep Learning Framework for Whole-Slide Image Segmentation and Analysis. TMI, 2020.
(.pdf) (abstract)

Histopathology tissue analysis is considered the gold standard in cancer diagnosis and prognosis. Given the large size of these images and the increase in the number of potential cancer cases, an automated solution as an aid to histopathologists is highly desirable. In the recent past, deep learning-based techniques have provided state of the art results in a wide variety of image analysis tasks, including analysis of digitized slides. However, the size of images and variability in histopathology tasks makes it a challenge to develop an integrated framework for histopathology image analysis. We propose a deep learning-based framework for histopathology tissue analysis. We demonstrate the generalizability of our framework, including training and inference, on several open-source datasets, which include CAMELYON (breast cancer metastases), DigestPath (colon cancer), and PAIP (liver cancer) datasets. We discuss multiple types of uncertainties pertaining to data and model, namely aleatoric and epistemic, respectively. Simultaneously, we demonstrate our model generalization across different data distribution by evaluating some samples on TCGA data. On CAMELYON16 test data (n=139) for the task of lesion detection, the FROC score achieved was 0.86 and in the CAMELYON17 test-data (n=500) for the task of pN-staging the Cohen's kappa score achieved was 0.9090 (third in the open leaderboard). On DigestPath test data (n=212) for the task of tumor segmentation, a Dice score of 0.782 was achieved (fourth in the challenge). On PAIP test data (n=40) for the task of viable tumor segmentation, a Jaccard Index of 0.75 (third in the challenge) was achieved, and for viable tumor burden, a score of 0.633 was achieved (second in the challenge). Our entire framework and related documentation are freely available at GitHub and PyPi.

Avinash Kori, Parth Natekar. Demystifying Brain Tumor Segmentation Networks: Interpretability and Uncertainty Analysis. Frontier in Computational Neuroscience, 2020.
(.pdf) (abstract)

The accurate automatic segmentation of gliomas and its intra-tumoral structures is important not only for treatment planning but also for follow-up evaluations. Several methods based on 2D and 3D Deep Neural Networks (DNN) have been developed to segment brain tumors and to classify different categories of tumors from different MRI modalities. However, these networks are often black-box models and do not provide any evidence regarding the process they take to perform this task. Increasing transparency and interpretability of such deep learning techniques are necessary for the complete integration of such methods into medical practice. In this paper, we explore various techniques to explain the functional organization of brain tumor segmentation models and to extract visualizations of internal concepts to understand how these networks achieve highly accurate tumor segmentations. We use the BraTS 2018 dataset to train three different networks with standard architectures and outline similarities and differences in the process that these networks take to segment brain tumors. We show that brain tumor segmentation networks learn certain human-understandable disentangled concepts on a filter level. We also show that they take a top-down or hierarchical approach to localizing the different parts of the tumor. We then extract visualizations of some internal feature maps and also provide a measure of uncertainty with regards to the outputs of the models to give additional qualitative evidence about the predictions of these networks. We believe that the emergence of such human-understandable organization and concepts might aid in the acceptance and integration of such methods in medical diagnosis.

2019

Prasanna Porwal et.al. IDRiD: Diabetic Retinopathy – Segmentation and Grading Challenge. Medical Image Analysis, 2019.
(abstract)

Diabetic Retinopathy (DR) is the most common cause of avoidable vision loss, predominantly affecting the working-age population across the globe. Screening for DR, coupled with timely consultation and treatment, is a globally trusted policy to avoid vision loss. However, implementation of DR screening programs is challenging due to the scarcity of medical professionals able to screen a growing global diabetic population at risk for DR. Computer-aided disease diagnosis in retinal image analysis could provide a sustainable approach for such large-scale screening effort. The recent scientific advances in computing capacity and machine learning approaches provide an avenue for biomedical scientists to reach this goal. Aiming to advance the state-of-the-art in automatic DR diagnosis, a grand challenge on “Diabetic Retinopathy – Segmentation and Grading” was organized in conjunction with the IEEE International Symposium on Biomedical Imaging (ISBI - 2018). In this paper, we report the set-up and results of this challenge that is primarily based on Indian Diabetic Retinopathy Image Dataset (IDRiD). There were three principal sub-challenges: lesion segmentation, disease severity grading, and localization of retinal landmarks and segmentation. These multiple tasks in this challenge allow to test the generalizability of algorithms, and this is what makes it different from existing ones. It received a positive response from the scientific community with 148 submissions from 495 registrations effectively entered in this challenge. This paper outlines the challenge, its organization, the dataset used, evaluation methods and results of top-performing participating solutions. The top-performing approaches utilized a blend of clinical information, data augmentation, and an ensemble of models. These findings have the potential to enable new developments in retinal image analysis and image-based DR screening in particular.

Avinash Kori, Manik Sharma. Dynamic Regularizer with an Informative Prior. Neural Information Processing Systems (NeurIPS). Under Review, 2019.
(abstract)

Regularization methods, specifically those which directly alter weights likeL1andL2, are an integral part of many learning algorithms. Both the regularizersmentioned above are formulated by assuming certain priors in the parameter spaceand these assumptions, in some cases, induce sparsity in the parameter space.Regularizers help in transferring beliefs one has on the dataset or the parameterspace by introducing adequate terms in the loss function. Any kind of formulationrepresents a specific set of beliefs:L1regularization conveys that the parameterspace should be sparse whereasL2regularization conveys that the parameter spaceshould be bounded and continuous. These regularizers in turn leverage certainpriors to express these inherent beliefs. A better understanding of how the prioreffects the behavior of the parameters and how the priors can be updated based onthe dataset can contribute greatly in improving the generalization capabilities ofa function estimator. In this work, we introduce a weakly informative prior andthen further extend it to an informative prior in order to formulate a regularizationpenalty, which shows better results in terms of inducing sparsity experimentally, when compared to regularizers based only on Gaussian and Laplacian priors. Theresulting regularizer is a non-convex superposition of theL1andL2penalties whichalso takes into account the notion of sparsity through its prior. Experimentally,we verify that a regularizer based on an adapted prior improves the generalizationcapabilities of any network. We illustrate the performance of the proposed methodon the MNIST and CIFAR-10 datasets

Avinash Kori, Kanti Kumari, Ganapathy Krishnamurthi. Zero Shot Learning for Multi-Modal Real Time Image Registration. Medical Image Computing and Computer Assisted Intervention (MICCAI). Under Review, 2019.
(abstract)

We present an unsupervised image registration framework, using a pre-trained deep neural network as a feature extractor. We refer this to zero-shot learning, due to non overlap between training and testing dataset (none of the network modules in the processing pipeline were trained specifically for the task of medical image registration). Highlights of our technique are: ( extbf{a}) No requirement of a training dataset ( extbf{b}) Keypoints i.e. locations of important features are automatically estimated ( extbf{c}) The number of key points in this model are fixed and can possibly be tuned as a hyperparameter. ( extbf{d}) Uncertainty calculation of the proposed, transformation estimates ( extbf{e}) Real-time registration of images. Our techniques were evaluated on BraTS, ALBERT and collaborative hospital Brain MRI data. Results suggest that the method proved to be robust for affine transformation models and the results are practically instantaneous, irrespective of the size of the input image.

Vismay Agrawal, Avinash Kori, Vikas Kumar Anand, Ganapathy Krishnamurthi. Structurally aware bidirectional unpaired image to image translation between CT and MR. Medical Image Computing and Computer Assisted Intervention (MICCAI). Under Review, 2019.
(abstract)

Magnetic Resonance (MR) Imaging and Computed Tomography (CT) are the primary diagnostic imaging modalities quite frequently used for surgical planning and analysis. General problem with medical imaging is that the acquisition process is quite expensive and time consuming. Deep learning techniques like generative adversarial networks(GAN) can help us to leverage the possibility of an image to image translation between multiple imaging techniques, which in turn helps in saving time and cost. These techniques will help to conduct surgical planning under CT with a feedback of MRI information. While previous studies have shown paired and unpaired image synthesis from MR to CT, image synthesis from CT to MR still remains a challenge. In this manuscript, we have implemented Generative Adversarial Network exploiting the cycling consistency and structural similarity between both CT and MR image modalities on a pelvis dataset, thus facilitating a bidirectional exchange of content and style between these image modalities. We have tested our proposed method by comparing it with the current state of the art image to image translation techniques like CycleGANs. Experimental results have shown that our network outperforms CycleGANs across various comparison metrics.

2018

Avinash Kori, Mehul Soni, B. Pranjal, Mahendra Khened, Varghese Alex, Ganapathy Krishnamurthi. Ensemble of Fully Convolutional Neural Network for Brain Tumor Segmentation from Magnetic Resonance Images. Medical Image Computing and Computer Assisted Intervention (MICCAI), 2018.
(.pdf) (abstract)

We utilize an ensemble of the fully convolutional neural networks (CNN) for segmentation of gliomas and its constituents from multimodal Magnetic Resonance Images (MRI). The ensemble comprises of 3 networks, two 3-D and one 2-D network. Of the 3 networks, 2 of them (one 2-D & one 3-D) utilize dense connectivity patterns while the other 3-D network makes use of the residual connection. Additionally, a 2-D fully convolutional semantic segmentation network was trained to distinguish between air, brain, and lesion in the slice and thereby localize the lesion the volume. Lesion localized by the above network was multiplied with the segmentation mask generated by the ensemble to reduce false positives. On the BraTS validation data (n = 66), the scheme utilized in this manuscript achieved a whole tumor, tumor core and active tumor dice of 0.89 0.76, 0.76 respectively, while on the BraTS test data (n = 191), our scheme achieved the whole tumor, tumor core and active tumor dice of 0.83 0.72, 0.69 respectively.

Aditya Bagari, Ashish Kumar, Avinash Kori, Mahendra Khened, Ganapathy Krishnamurthi. A Combined Radio-Histological Approach for Classification of Low Grade Gliomas. Medical Image Computing and Computer Assisted Intervention (MICCAI), 2018.
(.pdf) (abstract)

Deep learning based techniques have shown to be beneficial for automating various medical image tasks like segmentation of lesions and automation of disease diagnosis. In this work, we demonstrate the utility of deep learning and radiomics features for classification of low grade gliomas (LGG) into astrocytoma and oligodendroglioma. In this study the objective is to use whole-slide H&E stained images and Magnetic Resonance (MR) images of the brain to make a prediction about the class of the glioma. We treat both the pathology and radiology datasets separately for in-depth analysis and then combine the predictions made by the individual models to get the final class label for a patient. The pre-processing of the whole slide images involved region of interest detection, stain normalization and patch extraction. An autoencoder was trained to extract features from each patch and these features are then used to find anomaly patches among the entire set of patches for a single Whole Slide Image. These anomaly patches from all the training slides form the dataset for training the classification model. A deep neural network based classification model was used to classify individual patches among the two classes. For the radiology dataset based analysis, each MRI scan was fed into a pre-processing pipeline which involved skull-stripping, co-registration of MR sequences to T1c, re-sampling of MR volumes to isotropic voxels and segmentation of brain lesion. The lesions in the MR volumes were automatically segmented using a fully convolutional Neural Network (CNN) trained on BraTS-2018 segmentation challenge dataset. From the segmentation maps 64 × 64 × 64 cube patches centered around the tumor were extracted from the T1 MR images for extraction of high level radiomic features. These features were then used to train a logistic regression classifier. After developing the two models, we used a confidence based prediction methodology to get the final class labels for each patient. This combined approach achieved a classification accuracy of 90% on the challenge test set (n = 20). These results showcase the emerging role of deep learning and radiomics in analyzing whole-slide images and MR scans for lesion characterization.

Bhadra S. Kumar, Avinash Kori, Sundari Elango, V. Srinivasa Chakravarthy. Phase and Amplitude Modulation in a Neural Oscillatory Model of the Orientation Map. International Conference on Neural Information Processing (ICONIP), 2018.
(.pdf) (abstract)

The traditional approach to characterization of orientation maps as they were expounded by Hubel and Wiesel treats them as static representations. Only the magnitude of a neuron’s firing response to orientation is considered and the neuron with the highest response is said to be “tuned” to that response. But the neuronal response to orientation is a time-varying spike train and, if the response of an entire cortical area that potentially responds to orientations in a given part of the visual field is considered, the response must be considered as a spatio-temporal wave. We propose a neural field model consisting of FitzHugh-Nagumo neurons, that generates such a wave. Reflecting the dynamics of a single FitzHugh-Nagumo neuron, the neural field also exhibits excitatory and oscillatory regimes as an offset parameter is increased. We consider the question of the manner in which the input orientation is coded in the response of the neural field and discovered that two different codes − Amplitude Modulation and Phase Modulation − are present. Whereas for smaller offset values, when the model is in excitatory regime the orientation is coded in terms of amplitude, for larger offset values when the model is in the oscillatory regime, the orientation is coded in terms of phase.

Avinash Kori, Harikrishnan R. Comparative Study: Reinforcement Learning and Inverse Reinforcement Learning Algorithms. , 2018.
(abstract)

In this report we conducted a comparative study between value iteration, policy iteration and policy gradient methods, to study and analyze the efficiency of each algorithm in the discretized version of continuous state-space problem. We also conducted the analysis study between two different types of inverse reinforcement learning algorithms, namely linear programming and max-entropy. The rewards obtained by the agent using the aforementioned algorithms were used to train another RL agent whose performance is then studied using simulations. Value Iteration, policy iteration and policy gradient algorithms were compared on the basis of episodes taken for convergence, variance in episode rewards and the expected total reward. Comparison of IRL algorithms were done based on expected total reward and time taken for convergence. Tuning of hyperparameters were also conducted in case of both IRL algorithms.

Avinash Kori, Sai Saketh Chennamsetty, Mohammed Safwan K.P. and Varghese Alex. Ensemble of Convolutional Neural Networks for Automatic Grading of Diabetic Retinopathy and Macular Edema. International Symposium on Biomedical Imaging (ISBI), 2018.
(.pdf) (abstract)

In this manuscript, we automate the procedure of grading of diabetic retinopathy and macular edema from fundus images using an ensemble of convolutional neural networks.The availability of limited amount of labeled data to perform supervised learning was circumvented by using transfer learning approach. The models in the ensemble were pre-trained on a large dataset comprising natural images and were later fine-tuned with the limited data for the task of choice. For an image, the ensemble of classifiers generate multiple predictions, and a max-voting based approach was utilized to attain the final grade of the anomaly in the image. For the task of grading DR, on the test data (n=56), the ensemble achieved an accuracy of 83.9%, while for the task for grading macular edema the network achieved an accuracy of 95.45% (n=44).

2017

Mohammed Safwan, Sai Saketh Chennamsetty, Avinash Kori, Varghese Alex, Ganapathy Krishnamurthi. Classification of Breast Cancer and Grading of Diabetic Retinopathy & Macular Edema using Ensemble of Pre-trained Convolutional Neural Networks. , 2017.
(.pdf) (abstract)

In this manuscript, we present a deep learning based approach for detection and classification of medical conditions such as classification of breast cancer and grading of diabetic retinopathy and macular edema. The performance of a convolutional neural network is dependent on the architecture of the network, amount of training data and data pre-processing. Transfer learning is oft utilized in deep learning so as to counter the limited availability of high quality annotated data. Hence, we create an ensemble of pre-trained classifiers by making use of models with different topologies and data normalization schemes. In general, the variance associated with an ensemble of classifiers is lower compared to a single classifier and thus generalizes better on unseen data. An F1 score based model pruning technique was utilized for deciding the optimal number of classifiers in the ensemble. The proposed technique was tested on two separate biomedical image challenges, namely the (1) classification of breast cancer from histology images [BACH-2018] and (2) grading of diabetic retinopathy and macular edema from fundus images [IDRiD-2018]. On the histology data, our technique was adjudged jointly as the top performing algorithm while for the task of diabetic retinopathy grading, the technique was declared as the 4th best performing algorithm.

Krishna Chaitanya Kaluva, Mahendra Khened, Avinash Kori, Ganapathy Krishnamurthi. 2D-Densely Connected Convolution Neural Networks for automatic Liver and Tumor Segmentation. Medical Image Computing and Computer Assisted Intervention (MICCAI), 2017.
(.pdf) (abstract)

In this paper we propose a fully automatic 2-stage cascaded approach for segmentation of liver and its tumors in CT (Computed Tomography) images using densely connected fully convolutional neural network (DenseNet). We independently train liver and tumor segmentation models and cascade them for a combined segmentation of the liver and its tumor. The first stage involves segmentation of liver and the second stage uses the first stage's segmentation results for localization of liver and henceforth tumor segmentations inside liver region. The liver model was trained on the down-sampled axial slices , whereas for the tumor model no down-sampling of slices was done, but instead it was trained on the CT axial slices windowed at three different Hounsfield (HU) levels. On the test set our model achieved a global dice score of 0.923 and 0.625 on liver and tumor respectively. The computed tumor burden had an rmse of 0.044.

2016

Avinash Kori, Ganapathy Krishnamurthi, Balaji Srinivasan. Enhanced Image Classification With Data Augmentation Using Position Coordinates. , 2016.
(.pdf) (abstract)

In this paper we propose the use of image pixel position coordinate system to improve image classification accuracy in various applications. Specifically, we hypothesize that the use of pixel coordinates will lead to (a) Resolution invariant performance. Here, by resolution we mean the spacing between the pixels rather than the size of the image matrix.(b) Overall improvement in classification accuracy in comparison with network models trained without local pixel coordinates. This is due to position coordinates enabling the network to learn relationship between parts of objects, mimicking the human vision system. We demonstrate our hypothesis using empirical results and intuitive explanations of the feature maps learnt by deep neural networks. Specifically, our approach showed improvements in MNIST digit classification and beats state of the results on the SVHN database. We also show that the performance of our networks is unaffected despite training the same using blurred images of the MNIST database and predicting on the high resolution database.