Quality Assessment and Enhancement of Retinal Fundus Images using Deep Learning Methods

Abstract

In Ophthalmology, digital fundus photography is used to diagnose various ocular disorders like Cataract, Diabetic Retinopathy, Glaucoma, and Age-related macular degeneration etc. These diseases affect a substantial population worldwide. For effective medical assistance to such a huge number of patients, the current number of ophthalmologists are inadequate. Telemedicine and computer-aided diagnosis systems are the potential solutions to address the lack of required number of medical doctors. In addition, advances in image acquisition devices led to change in the set-up of fundus image acquisition from a fixed position to portable devices such as D-EYE, Panoptic ophthalmoscope. It allows medical doctors to capture, analyze, and share the fundus images easily. However, images captured by such portable devices are more vulnerable to various types of distortions than by devices in controlled set-up. A trustworthy diagnosis solely relies upon the quality of the fundus image. This thesis aims to design and develop novel approaches using deep learning methods to ensure the quality of retinal images. To this end, a careful study of the state-of-the-art in the field of fundus IQA research is done to determine the important limitations in these methods. It has been found that the generalizability of the current work is limited, as the existing quality models are developed and evaluated with data-sets built with less subjective inputs. Also, most of the published works are for binary classification methods: Good and Poor. However, in practice, fundus images exist that neither falls into the good nor in the poor class. Such images are often termed as fair or average quality images. Our first work aims at addressing these limitations with the following two contributions. First, a new fundus image quality assessment (FIQuA) data-set is presented, containing 1500 fundus images with three classes of quality: Good, Fair, and Poor. Also, for each image, subjective scores (in the range [0-10]) were collected for six local quality p rameters, including structural and generic properties of the fundus images. Second, a new multivariate regression-based neural network model is proposed to predict the fundus image quality. The proposed model consists of two individually trained blocks. The first block consists of four pre-trained models, trained against the subjective scores for the six quality parameters, and aims at deriving the optimized features for classification. Next, the optimized features from each of the four models are ensembled together and transferred to the second block for final classification. The proposed model achieves a strong correlation with the subjective scores, with the values 0.941, 0.954, 0.853, and 0.401 obtained for SROCC, LCC, KCC, and RMSE respectively. Its classification accuracy is 95.66% over the FIQuA data-set, and 98.96% and 88.43% respectively over the two publicly available data-sets DRIMDB and EyeQ. Furthermore, a fair quality fundus image indicates that the quality of the given image does not satisfy all the necessary expectations, but at the same time, the image may support a diagnosis in some contexts. Such images contain a small amount of distortions but still be usable for diagnosis by medical doctors. At the same time, these images might lead to wrong diagnostic results from automated diagnosis systems. Additionally, fair category images also indicate the scope of enhancement. A study at UK BioBank indicates that more than 25% fundus images neither fall into the category of good quality nor poor. Therefore an efficient fundus image enhancement is required to maximally recover such fair quality fundus images. The retinal image enhancement methods are necessary to ensure diagnostic reliability and save time and effort for reacquiring the images. The enhancement works reported earlier are developed for distortions, mostly caused by additive white Gaussian and salt-and-pepper noises. However, this poses a significant limitation about the applicability of these methods as occurrences of such distortions are least likely. In our next work, the five most common distortions in fundus images are identified and algorithms are proposed to create the distortions resembling the same. After that, a residual dense connection based UNet (RDC-UNet) architecture is proposed for the enhancement task. The residual dense connections incorporated in the UNet, effectively captures both the local and global information from the images beneficial for the enhancement task. The RDC-UNet is trained individually for each of the five distortions and then applied to the synthetic degraded fundus images. The experimental results show that visual quality and the quantitative results, on an average, are 25% better than the state of the art methods reported in the literature. Furthermore, in case of naturally degraded images, the type of distortion is not known apriori. Also, multiple such distortions can be present at a time. To address this challenge, an ensemble model architecture is proposed using the RDC-UNet trained individually for each degradation.Experiments conducted over naturally degraded fundus images demonstrate that the proposed model effectively enhances the visual quality of fundus images. The proposed approach will have a high impact in data-driven technology development in various fields, not limited only to enhancement. For a given application, synthetic data similar to the actual ones can be generated, for proof of concept validation of the proposed algorithm, and fine-tuning can be done at a later stage once the actual data is available.