We propose a non-parametric method of jointly estimating non-rigid motion and the underlying image without the assumption of motion smoothness. We model non-rigid motion as local linear translation, which is equivalent to convolution with 1-sparse kernels. We then pose the non-rigid motion recovery problem as a sparse blind deconvolution problem. Our reconstruction results demonstrate that non-rigid motion can be well approximated as local translation motion using the proposed method. The proposed formulation can also be viewed as a generalization of locally low rank reconstruction.