Manual annotation of muscle is still one of the most time-consuming steps in skeletal muscle MRI research. In this study we have investigated three aspects of automated muscle annotation using deep convolutional networks. First, we directly compare five different network architectures. Second, we compare the effect of providing various input data all based on Dixon imaging. Third, we investigate the effect of the amount of training data provided to the network. In summary we found that UNet-like convolutional networks allow for accurate and precise annotation of calf muscle in 2D and 3D and that the data provided is the strongest predictor of success.