Different brain MRI contrasts represent different tissue properties and are sensitive to different artifacts. The relationship between different contrasts is therefore complex and nonlinear. We developed a deep convolutional network that learns the mapping between different MRI contrasts. Using a publicly available dataset, we demonstrate that this algorithm accurately transforms between T1- and T2-weighted images, proton density images, time-of-flight angiograms, and diffusion MRI images. We demonstrate that these transformed images can be used to improve spatial registration between MR images of different contrasts.