Incorporating TGV-regularization to accelerated model-based parameter-quantification can lead to improved image quality, however, adds non-differentiability to the problem which poses a problem for commonly used first order optimization methods like non-linear Conjugate-Gradient. The proposed method overcomes this limitation by handling the problem within a Gauss-Newton-framework and applying a Primal-Dual-algorithm to solve the inner TGV-regularized problem. Numerical simulations exhibit high agreement to references for four different parameter mapping problems up to 18-fold acceleration. In-vivo results for T1-VFA and T2-MESE models strengthen these findings. The proposed method offers huge acceleration potential for model-based parameter-quantification with similar quantification quality as fully sampled data.