The movement of animals is a central component of their behavioural strategies. Statistical tools for movement data analysis, however, have long been limited, and in particular, unable to account for past movement information except in a very simplified way. In this work, we propose MoveFormer, a new step-based model of movement capable of learning directly from full animal trajectories. While inspired by the classical step-selection framework and previous work on the quantification of uncertainty in movement predictions, MoveFormer also builds upon recent developments in deep learning, such as the Transformer architecture, allowing it to incorporate long temporal contexts. The model predicts an animal’s next movement step given its past movement history, including not only purely positional and temporal information, but also any available environmental covariates such as land cover or temperature. We apply our model to a diverse dataset made up of over 1550 trajectories from over 100 studies, and show how it can be used to gain insights about the importance of the provided context features, including the extent of past movement history. Our software, along with the trained model weights, is released as open source.