Landmark detection based on deep neural networks has achieved state-of-the-art performance in natural image analysis. However, it is challenging to detect anatomical landmarks from medical images, due to limited data. Here, we propose a real-time large-scale landmark detection method with limited training data. We train our model with image patches and test it with the entire image, inspired by fully convolutional networks. Also, we develop a weighted loss function in our model to increase the correlations between image patches and their nearby landmarks. The experimental results of detecting 1741 landmarks from brain MR images demonstrate the effectiveness of our method.