MRI-driven computational constitutive modeling can be used to obtain subject-specific myocardial passive stiffness. Verifying the accuracy and precision of this technique requires overcoming the challenge of obtaining ground-truth in vivo myocardial stiffness. We developed a controllable in vitro diastolic filling setup which incorporates a soft heart phantom of known myocardium-mimicking mechanical stiffness and MRI properties. Using the setup we demonstrate that uncertainties in quantifying cardiac reference geometry can lead to errors in estimating myocardial passive stiffness. The in vitro setup is designed to enable us to achieve our overarching goal: to extensively validate in vivo MRI-based myocardial passive stiffness estimation.