Cerebral small vessel disease(CSVD) is the most common cause of vascular cognitive impairment. Early diagnosis and intervention could prevent patients from progressing to dementia rapidly. DTI deterministic tractography and structural network analysis based on graph theory were utilized to explore the difference of structural connectome between CSVD patients and normal individuals. The results demonstrated the disruption of structural connection at global level and reduction of network efficiency at nodal level. Especially, the decrease of nodal efficiency in the right cuneus gyrus was associated with the mild memory loss in CSVD patients. Our results may provide crucial clues to therapeutic interventions.