Geometric distortions caused by off-resonant spins are a major issue in EPI based functional and diffusion imaging. We present a novel approach to the problem of geometric distortions. An extension of the model-based, algebraic hybrid-SENSE reconstruction method in combination with a known fieldmap, calculated from blip up/down scans, allows for correction of off-resonance effects during the reconstruction. This enables a joint blip up/down reconstruction that significantly reduces g-factor penalty if the blip-down trajectory is chosen to fill in missing k-space samples from the blip-up scan. The resulting high SNR images are automatically eddy-current corrected and geometric distortions are minimized.