Image quality plays a vital role in automated pipelines for medical image processing. Automated tools have thus been developed to detect low-quality images and ensure reliability of downstream results. These tools, however, often rely on image processing algorithms that can be sensitive to certain image features. In this study, we investigate the reproducibility of image quality measures provided by the open source image quality control tool MRIQC with respect to different scan setups. Results show that the reproducibility of some IQ measures is linked to the variation in the scan setup while for others it is less dependent on it.