We used diffusion spectrum imaging (DSI) to investigate the relationships between brain structure and behavior in children with specific learning disabilities (SLD). The correlation between reading comprehension scores and the DSI indices was found in corpus callosum. The correlation between Chinese character recognition and the DSI indices was found in cingulate and corpus callosum. The correlation between tone awareness scores and the DSI indices was found in cingulate, superior frontal gyrus and corpus callosum. In summary, SLD not only had difficulty reading and spelling individual words but also more likely to have poorer phonological awareness.