The water sideband artifact is a major obstacle to proton MR spectroscopic imaging (1H-MRSI) without water suppression (WS). This work presents a novel method to remove the sideband artifacts from non-WS MRSI data, characterized by the use of a reference-based parametric model to represent the artifacts. Our method obtains the reference signal from two auxiliary scans and subsequently estimates the sideband signals from a particular MRSI data. The proposed method has been validated using both phantom and in vivo experimental data, demonstrating that it can effectively remove the sideband artifacts without introducing spectral distortion. This method is expected to be useful for many non-WS MRSI studies.