Non-uniformly sampling is an effective way to accelerate high-dimensional magnetic resonance spectroscopy (MRS). The spectrum is usually reconstructed with proper prior knowledge. In this work, we exploit the intrinsic N-D exponential signals of multi-dimensional MRS to reconstruct the spectrum. A low rank tensor representation of multi-dimensional MRS and the exponential structure of the associated factors are simultaneously explored. Results on 3-D MRS data shows that the proposed method can faithfully reconstruct the spectrum from a small amount of measurements, allowing a significant reduction of acquiring time in real applications.