To eliminate technical variability in quantitative diffusion imaging applications, the systematic bias in diffusion weighting gradients should be corrected. The major source of this bias is the system-specific spatial gradient non-linearity (GNL) that can be rectified using gradient design information independent of the scanned object. This study characterizes the residual sources of nonuniform diffusion weighting introduced by imperfect object-dependent B0 shimming. Using controlled de-shim gradients, we show that an imperfect shim leads to systematic offsets of the otherwise symmetric GNL profile relative to the isocenter. The empiric strategies are proposed to mitigate the shim-induced errors in ADC measurements.