Atrophy of the spinal cord is known to occur in multiple sclerosis (MS). To measure such atrophy, the mean upper cervical cord area (MUCCA) can be assessed. We tested five different (semi-)automated spinal cord segmentation methods (SCT-PropSeg, SCT-DeepSeg, ITK-SNAP, NeuroQLab, Xinapse JIM) in terms of their reproducibility, robustness, and the influence of lesions on the segmentations. MUCCA from all scanners was highly reproducible within-scanner, but not between-scanner or between-methods. The presence of lesions in the upper cervical cord did not affect the accuracy of MUCCA measurements in any of the methods tested.