Deep Learning from Electronic Medical Records using Attention-Based Cross-Modal Convolutional Neural Networks

Bing Leung Patrick Cheung1, Deborah Dahl

  • 1Philips Research North America

Details

15:15 - 15:30 | Tue 6 Mar | Treasure Island ABC | TuBT1.5

Session: BHI Session # 4 – Deep Learning and Decision Support Systems

Abstract

An attention-based cross-modal convolutional neural network (AXCNN) is introduced for predictive analytics in healthcare from electronic medical records (EMRs). It is composed of sub-modules with specialized deep learning model architectures at the lower layer to extract feature representations from input data consisting of patient background information, medical codes, vital signs and lab results followed by a cross-modal convolution module to integrate the information between them. In addition, each sub-module is associated with an attention module which provides an insight on those input variables that are learned to attend by the prediction model. The effectiveness of this deep learning model is demonstrated in the context of hospital readmission prediction using EMRs from 6730 heart failure patients from a large healthcare system in the U.S. The empirical results show that the AXCNN model improves the AUC score by 0.0254 compared with the convolutional neural network that does not take input data types into consideration.