Humans interact in groups through various perception and action channels. The continuity of interaction despite a transient loss of perceptual contact often exists and contributes to goal achievement. Here, we study the dynamics of this continuity, in two experiments involving groups of participants (N = 7) synchronizing their movements in space and in time. We show that behavioural unison can be maintained after perceptual contact has been lost, for about 7s. Agent similarity and spatial configuration in the group modulated synchronization performance, differently so when perceptual interaction was present or when it was memorized. Modelling these data through a network of oscillators enabled us to clarify the double origin of this memory effect, of individual and social nature. These results shed new light into why humans continue to move in unison after perceptual interruption, and are consequential for a wide variety of applications at work, in art and in sport.