Tissue parameter estimation often gives noisy parameter maps due to noise in the signal data. In this work, we improve parameter mapping by incorporating noise reduction and uncertainty estimation with Bayesian Deep Image Prior. We implement task-specific loss functions for different applications. We test our method by estimating T1, T2, and ADC with synthetic- and in-vivo MRI data. Our method results in denoised tissue parameter maps, with associated error estimates. Our method is easy to implement, does not require any training data, and is easy to customize for different applications in parameter mapping.