Model-based accelerated imaging techniques enable high scan time reductions while maintaining high image quality. These techniques rely on the ability to accurately estimate the imaging model. This model can be extended to include information beyond physical limits, such as high-resolution phase information to promote conjugate symmetry or information of voxels without signal for a stronger image prior. Thus, we propose a deep learning approach to estimate the imaging model with latent coil maps. Furthermore, we jointly train this latent map estimator with a deep-learning-based reconstruction using adversarial loss, and we demonstrate the effectiveness of this approach in volumetric knee datasets.