Unsupervised deep neural networks (DNN) are successfully employed to predict deformation-fields in neuroimaging studies. Bayesian DNN models enable safer utilization of DNN methods in neuroimaging studies, improve generalization and enable assessment of uncertainty in the predictions. We propose a non-parametric Bayesian approach to estimate the uncertainty in DNN-based algorithms for brain MRI deformable registration. We demonstrated the added-value of our Bayesian registration framework on the brain MRI (LPBA40) dataset compared to state-of-the-art VoxelMorph DNN. Further, we quantified the uncertainty of the registration and assessed its correlation with the out-of-distribution data.