Visual cortex functionality in the blind has been shown to shift away from sensory networks toward task-positive networks that are involved in top-down modulation. However, how such modulation is shaped by experience and reflected behaviorally remains unclear. Using blood-oxygenation-level-dependent functional MRI with a sensory substitution task, we found that top-down visual cortex activity negatively correlates with duration of blindness and reaction time. Our results suggest that alterations in top-down brain activity due to visual deprivation progress as a function of time. Furthermore, the degree of top-down activity in the visual cortex may reflect the speed of performance during sensory substitution.