Many imaging applications, such as dynamic imaging and multi-contrast imaging, involve the acquisition of a sequence of images. This work addresses the underlying image reconstruction problem by incorporating priori information such as partial separability, image sparsity, and manifold structure jointly to enable high-quality image reconstruction from highly sparse data. To this end, we propose a new deep learning-based framework that enforces those constraints effectively and consistently. The proposed method has been validated using multi-contrast imaging data and produced impressive results. The image reconstruction framework can be extended for incorporating additional constraints and/or solving other sequential image reconstruction problems.