Multi-modal MR images are widely utilized to overcome the shortcomings of single modalities and pursue accurate image-based diagnoses. However, multi-modal MR imaging takes a longer time. For automated diagnosis, misalignment between modalities brings extra problems. To address these issues, we propose a new strategy to speculate the modality information by distillation-based knowledge transfer. Experiments on breast lesion segmentation confirm the feasibility of the proposed method. Networks trained with our method and single-modal MR image inputs can partially recover the breast lesion segmentation performance of models trained using two-modal MR images.