Deep learning approaches for Alzheimer's disease (AD) classification have primarily focused on modelling the structural changes associated with the condition, neglecting the changes in functional brain dynamics. These functional changes may be detectable earlier than structural atrophy providing an avenue for earlier diagnosis and treatment. We therefore proposed a convolutional neural network combined with a long short-term memory unit to decode fMRI signals. The model was able to classify AD from healthy control with a balanced accuracy of 0.69. Whilst there is room to improve network performance, the study already provides promising insights into the possibilities of resting-state fMRI classification.