Automatic detection and classification of brain tumors was performed using deep learning based on conventional MRI and clinical information. A total of 441 patients where included: 202 patients with high grade glioma and 239 patients with brain metastases. Classification was performed using resnet34 architecture. The input data for classification were FLAIR images, post contrast T1W images and patients’ clinical information. Classification results showed high accuracy=89%, specificity=91% and sensitivity=86%. For lesion localization the mean intersection over union (IoU) score was 0.64±0.17. Our results indicate the promising potential of a deep learning approach for automatic non-invasive diagnosis of patients with brain tumors.