In human brain, there are many cognitive demands sharing similar brain regions including working memory, attention, mathematical calculation and reasoning, and problem solving. These multiple-demand (MD) brain regions mainly involve the frontal and parietal lobes including the posterior-lateral frontal, dorso-medial frontal, and mid-parietal cortices. Previous studies with limited age ranges of the participant population reported the total brain volume was highly correlated with Gf and inconsistent findings of white matter tracts related with Gf. Fewer research explores the life-span patterns of Gf with gray and white matter and no previous study investigated the relationship of left and right hemispheres with Gf. The present study aimed to probe the life-span relationship between gray matter volume and white matter tracts connecting the MD regions and Gf in an adult life span large cohort of 603 normal participants from 18 to 88 years old. We also further compared the contributions of the left and right hemispheres to Gf. We hypothesized that gray matter volume and white matter tracts connecting the MD network may reflect age-related changes of Gf, and that Gf is a lateralized complex function. The novelty of our findings is that we examined larger normal adult population across 18 to 88 years old and found that both gray matter volume and white matter tract integrity in the MD regions are the neural substrates of Gf, reflecting the life-span aging patterns of Gf. We further examined the relationship of left and right hemispheres with Gf and found that left and right MD regions showed similar patterns of correlations with Gf scores. The age-related decrease of gray matter volume and tract integrity in the MD network is associated with the reduced functions of multiple-demand cognitive abilities reflected by Gf scores.