Meeting Banner
Abstract #4551

Video C3D features learned by deep network correlate with functional MRI signal variation associated with the video

Xu Chen1, Jason Langley1, Sujoy Paul2, Tahmida Mahmud2, Amit K Roy-Chowdhury2, Aaron Seitz3, and XiaoPing Hu1,4

1Center for Advanced Neuroimaging, UC Riverside, Riverside, CA, United States, 2Dept. of Electrical & Computer Engineering, UC Riverside, Riverside, CA, United States, 3Dept. of Psychology, UC Riverside, Riverside, CA, United States, 4Dept. of Bioengineering, UC Riverside, Riverside, CA, United States

To gain further insights into the mechanisms of deep network learning from the perspective of brain imaging, we compared spatio-temporal features of video segments extracted via a 3-dimensional convolutional network (3D ConvNets) with video representations in human brain characterized by functional MRI signal variation when viewing video segments. We found correlations between C3D features and fMRI signal variation in brain regions selectively activated by video segments after the optimization of time lag due to the hemodynamic response function (HRF). Distinct activation patterns were also revealed by functional MRI for video segments classified as different classes of activity.

This abstract and the presentation materials are available to members only; a login is required.

Join Here