Siamese Network is an artificial neural network that has been used in small sample sets multi-class classification studies. This study identified major depressive disorder (MDD), bipolar disorder (BD), and schizophrenia (SZ) based on combined gray matter, white matter and cerebrospinal fluid using Siamese Network. The participants included four groups: MDD (n = 102), BD (n = 44), SZ (n = 114), and healthy controls (n = 103). We found Siamese Network achieved improved performance than the multilayer perception network with different numbers of features. We achieved a classification accuracy of 46.06% and Macro F1 of 41.47% for this multi-class identification.