The isocitrate dehydrogenase (IDH) gene mutation status is a prognostic biomarker for gliomas. As an alternative approach to the invasive gold standard of IDH mutation detection, radiogenomics approaches using features of MRI showed promising results in the same task. Here, we proposed an approach to predict the IDH status based on the image and morphology features extracted using contrastive learning. We then constructed a large patient graph based on the extracted features, which could predict the IDH mutation using graph neural networks. The results showed the proposed method outperforms the classifiers which leverage either image or morphology features only.