Prediction of response to neoadjuvant systemic therapy for triple-negative breast cancer is important for patient management. Here we constructed a deep learning convolutional and recursive neural network ensemble for early prediction of pathologic complete response utilizing pre-treatment DCE and DWI breast MRIs. Images from 135 patients were partitioned into training/validation/testing groups with the ratio of 80/20/35. For the testing group, the network achieved an accuracy of 69%, with the sensitivity of 75% and specificity of 63%. The area under the receiver operating characteristic curve was 0.68.