Respiratory induced organ motion reduces radiation delivery accuracy of radiotherapy in thorax and abdomen. MR-guided-radiotherapy (MRgRT) is capable of continuous MRI acquisition during treatment. However, the latency due to MRI acquisition and reconstruction, the detection of tumor position change, and the interaction with multileaf collimator (MLC) have been identified as the major challenges for real-time MRgRT. In this study, we proposed a deep-learning based 3D motion prediction technique to predict liver motion from volumetric dynamic MR images. Our algorithm showed promising results (< 0.3 cm prediction error on average) , suggesting its possibility of real-time motion tracking in the future MRgRT.