Dynamic contrast-enhanced magnetic resonance (DCE-MR) imaging is an important clinical tool for investigating the cerebrovascular microcirculatory systems including the blood-brain barrier through estimation of perfusion and permeability maps. However, a vascular function is needed to generate these maps. The estimation is usually performed manually, potentially leading to error, and is also time-consuming. In this work, we designed a deep learning model that leverages the temporal and spatial information from a time series of DCE-MR images to estimate the vascular function automatically. Our model was able to generalize well for unseen data and achieved good overall performance.