CEST MRI is sensitive to dilute metabolites with exchangeable protons, allowing tissue characterization in diseases such as acute stroke and tumor. CEST quantification using multi-Lorentzian fitting is challenging due to its strong dependence on image SNR, initial values and boundaries. Here we proposed an Image Downsampling Expedited Adaptive Least-squares (IDEAL) fitting algorithm that quantifies CEST images based on initial values from multi-Lorentzian fitting of iteratively less downsampled images. The IDEAL fitting provides smaller coefficient of variation and higher contrast-to-noise ratio at a faster fitting speed compared to conventional fitting. It revealed pronounced CEST contrasts in tumors which were not found using conventional method. The proposed method can be generalized to quantify MRI data where SNR is suboptimal.