Left and right ventricle segmentation is an important step in quantitative analysis of cardiac MR images. Convolutional neural networks (CNN) have shown great improvement and are quickly becoming the mainstream methods. One challenge in cardiac MRI segmentation from short-axis images is the variability of the imaging views and the fact that CNN is not rotation-invariant. To address this issue, we trained a view-independent network and further improved its performance with a rotation-based testing augmentation. Consistent improvement in performance was obtained as measured by Dice scores and visual contour quality.