We proposed a new convolutional neural network (CNN) to generate high resolution (HR) MR images from highly down-sampled MR images, incorporating HR images in another contrast. Anatomical information from another HR images and adversarial loss functions allowed the proposed model to restore details and edges clearly from the down-sampled images, proved in normal and brain tumor regions. Pre-training with a public database improved performance in real human applications. The proposed methods outperformed several CS algorithms in both pseudo-k-spaces from public data and real k-spaces from human brain data. CNNs can be a good alternative for accelerating routine MRI scanning.