High-resolution magnetic resonance image (MRI) are favorable by clinical application thanks to its detailed anatomical information. However, high spatial resolution typically comes at the expense of longer scan time, less spatial coverage, and lower signal to noise ratio (SNR). Single Image Super-Resolution (SISR), a technique aimed to restore high-resolution (HR) details from one single low-resolution (LR) input image, has been improved dramatically by the recent invention of deep Generative Adversarial Networks(GAN). In this paper, we introduce a new neural networks structure, 3D Densely Connected Super-Resolution GAN (DSRGAN) to realistic restore HR features of structural brain MR images. Through experiments on a dataset with 1,113 subjects, we demonstrate that our network outperforms bicubic interpolation in restoring 4x resolution-reduced images.