The advent of fetal magnetic resonance imaging has provided innovative approaches to study in-vivo brain development in the womb. One of the major challenges in the quantification of fetal brain growth and development is regional and tissue-specific segmentation. An automated brain localization algorithm can reduce the time and facilitate accurate segmentation. In this study, we propose an ultra-fast and robust method for fetal brain localization from SSFSE anatomical images using a minimally modified object localization algorithm called You Only Look Once (YOLO). YOLO provides not only the enhanced accuracy of brain localization by differentiating brain from maternal tissues but also fast computation time for brain detection compared to the other algorithms.