An auxiliary CHD diagnostic system based on multi-view and multi-modal transthoracic echocardiograms

Congenital heart disease (CHD) is one of the most common congenital anomalies worldwide, which brings a heavy health and financial burden to patients. Early CHD screening and treatment can significantly improve children’s prognosis and quality of life. However, inexperienced sonographers often face difficulties in recognizing CHD through transthoracic echocardiogram (TTE) images. An auxiliary CHD screening system that allows inexperienced sonographers and general practitioners to perform TTE in a simple and easy-to-use way, thereby improving the CHD screening rate and scope, is urgently needed.

An Anhui Medical University, National Center for Children’s Health and National Research Institute for Family Planning co-team have developed a new CHD detection system to identify the TTE cardiac views, integrate information from various views and modalities, visualize the high-risk region, and predict the probability of the subject being normal, atrial septal defect (ASD), or ventricular septal defect (VSD).

“Although multi-modal and multi-view TTEs pose challenges to exploiting the meaningful features from this high-dimensional data, they contain rich information indicating cardiac status worth mining,” said Wanqing Xie, who led the research. Xie is a professor of biomedical engineering at Anhui Medical University.

The team met the challenge by creating a hierarchical network structure. The model first recognized the two modalities (2D and Doppler TTE) and identified the cardiac views of TTEs: apical four chamber (A4C), subxiphoid long axis view (SXLAX) of two atria, parasternal long-axis view (PSLAX) of the left ventricle, parasternal short-axis view (PSSAX) of aorta, and suprasternal long-axis view (SSLAX). Then, the model embedded the features per view for each modality based on the backbone network ResNet50. After the basic feature embedding module, the model fused the embeddings of five views and then merged the information from two modal TTEs. Finally, the prediction of each subject was calculated from the classifier, and the visualization of high-risk regions for each child was generated using the Grad-CAM strategy.

After the child completed the TTE examination, Xie’s auxiliary CHD diagnostic system automatically analyzed the TTE images and computed the probability of each subject being normal, having ASD, or having VSD. The researchers demonstrated that the model accurately identified the children with CHD by integrating the TTEs with multiple views and modalities. The results suggested that the model has the potential to facilitate and improve widespread screening and the distinguishing of CHD subtypes in children.