No study to date has performed a rigorous analysis of the relative contributions of multi-modal imaging data including the brain’s functional (FC) and structural connectivity (SC) in the task of classifying high and low adapting MS patients for a deeper understanding of the connectome-level mechanism contributing to variability in MS-related impairment. We built a machine learning based ensemble model that can accurately classify MS patients as high and low adapters (AUC> 0.626). We observed that SC and FC networks can be used to identify the most discriminative regions and to accurately classify MS patients regarding their impairment level.