A pipeline based on the XCAT phantom, the JEMRIS software for simulating the MR signal and a commercial reconstruction pipeline has been set-up for simulating realistic brain MRI images. Using this pipeline, an anatomically variable brain MRI population is simulated across age and gender. Anatomical variation is generated by means of changing individual brain sizes, and cortical gray matter volumes to mimic aging brain. MS lesions are simulated to mimic diseased brain as well. Significant contrast is generated across detailed brain structures. The commercial reconstruction pipeline increased the realism of simulated data.