The rupture of unstable or vulnerable atherosclerotic plaque is the major cause of ischemic stroke. Manual analysis of the vessel wall and plaque is labor-intensive and experience dependent. The purpose of this study is to develop an automatic method to segment the vessel wall and lumen contour for quantitative measurement. In this work, a CNN architecture for fully automated segmentation of arterial lumen and vessel wall on MR vessel wall images was developed and evaluated on ischemic stroke patients. In conclusion, we proposed a good performance automatic analysis method for the vessel wall and is important for plaque analysis.