Deep learning denoising using a convolutional neural network (DnCNN) is proposed to accelerate the acquisition of diffusion-weighted imaging (DWI) data in patients with rectal tumors. 4-fold acceleration was achieved by reducing the number of averages from 16 to 4 and applying the DnCNN to denoise the data. The DnCNN was trained using pairs of noisy (1 average) and reference (16 averages) images from 92 patients. The trained network was then tested with the data from 6 patients and the results were evaluated qualitatively by radiologists. DnCNN represents a powerful approach to improve efficiency of DWI in patients with rectal cancer.