The representational nature of spatio-temporal recurrent processing in visual object recognition

Abstract

SummaryThe human brain orchestrates object vision through an interplay of feedforward processing in concert with recurrent processing. However, where, when and how recurrent processing contributes to visual processing is incompletely understood due to the difficulties in teasing apart feedforward and recurrent processing. We combined a backward masking paradigm with multivariate analysis on EEG and fMRI data to isolate and characterize the nature of recurrent processing. We find that recurrent processing substantially shapes visual representations across the ventral visual stream, starting early on at around 100ms in early visual cortex (EVC) and in two later phases of around 175 and 300ms in lateral occipital cortex (LOC), adding persistent rather than transient neural dynamics to visual processing. Using deep neural network models for comparison with the brain, we show that recurrence changes the feature format in LOC from predominantly mid-level to more high-level features. Finally, we show that recurrence is mediated by four distinct spectro-temporal neural components in EVC and LOC, which span the theta to beta frequency range. Together, our results reveal the nature and mechanisms of the effects of recurrent processing on the visual representations in the human brain.