Multi-Spectral Imaging (MSI) enables MRI near metallic implants, but suffers from prolonged scan times. Model-based reconstruction accelerates MSI by enforcing a signal model along the spectral dimension to reduce the number of unknowns in image reconstruction. The previous signal model assumes that spins in one voxel have the same off-resonance frequency, which tends to fail where the off-resonance field changes rapidly. Here we propose a more flexible MSI signal model that allows multiple frequencies within a voxel, and demonstrate improvements with both simulated and in-vivo data. 3x net additional acceleration above partial-Fourier and parallel-imaging alone (20x in total) was achieved.