Generating Target/non-Target Images of an RSVP Experiment from Brain Signals in by Conditional Generative Adversarial Network

Abstract

Understanding human brain activities and their associations with sensory stimuli is an important area of brain research. We present in this paper the reconstruction of target and nontarget images from Electroencephalography (EEG) signals collected in a Rapid serial visual presentation (RSVP) experiment. We proposed a novel model based on conditional Generative Adversarial Networks (cGAN), which includes a generator to generate target/nontarget images from input EEG epochs and discriminator to discriminate true images from the generated images. We showed the performance of image generation of the proposed cGAN model based EEG or EEG plus noise as input. We further demonstrated how we could use the trained model to examine the associations between target/nontarget images and their induced EEG patterns.