Measuring the Performance of Computer Vision Systems in Malaria Studies

Abstract

Digital image processing-computer vision (DIP-CV) systems are used to automate malaria diagnosis through microscopy analysis of thin blood smears. Some variability is observed in the experimental design to evaluate the statistical measures of performance (SMP) of such systems. The objective of this work is assessing good practices when using SMP to evaluate DIP-CV systems for malaria diagnosis. A mathematical model was built to characterize diagnosis using DIP-CV systems and used to obtain curve families showing the relationships among various SMP of these systems, both using theoretical equations and computer simulation. Curve families showing (a) the relationships among the minimum number of positive erythrocytes (RBCs) to be observed, the per object (RBC) sensitivity and the probability to detect at least one positive, (b) per specimen sensitivity vs. total number of RBCs observed for a typical per object sensitivity and a range of parasite densities (c) per object positive predictive value vs. per object specificity for a typical per object sensitivity and various parasite densities. When determining the per specimen sensitivity, the parasite density p showed to have more influence on the number of RBCs that must be analyzed than the per object sensitivity. Measuring p accurately depends heavily upon the per object positive predictive value of the classifier. For low p values, this would require very high per object specificity and a high enough value of observed RBCs to measure this accurately.