Non-rigid image registration is a fundamental procedure for the quantitative analysis of brain images. The goal of non-rigid registration is to obtain the smooth deformation field that can build anatomical correspondences among two or more images. Conventional non-rigid registration methods require iterative optimization with careful parameter tuning, which is less flexible when dealing with the diverse data. Therefore, we propose a two-stage deep network to directly estimate the deformation field between an arbitrary pair of images. This method can tackle various registration tasks, and is consistently accurate and robust without parameter tuning. Thus, it is applicable to clinical applications.