Upscaling dispersion, mixing, and reaction processes from the pore to the Darcy scale is directly related to the understanding of the dynamics of pore-scale particle velocities, which are at the origin of hydrodynamic dispersion and non-Fickian transport behaviors. With the aim of deriving a framework for the systematic upscaling of these processes from the pore to the Darcy scale, we present a detailed analysis of the evolution of Lagrangian and Eulerian statistics and their dependence on the injection condition. The study is based on velocity data obtained from computational fluid dynamics simulations of Stokes flow and advective particle tracking in the three-dimensional pore structure obtained from high-resolution X-ray microtomography of a Berea sandstone sample. While isochronously sampled velocity series show intermittent behavior, equidistant series vary in a regular random pattern. Both statistics evolve toward stationary states, which are related to the Eulerian velocity statistics. The equidistantly sampled Lagrangian velocity distribution converges on only a few pore lengths. These findings indicate that the equidistant velocity series can be represented by an ergodic Markov process. A stochastic Markov model for the equidistant velocity magnitude captures the evolution of the Lagrangian velocity statistics. The model is parameterized by the Eulerian velocity distribution and a relaxation length scale, which can be related to hydraulic properties and the medium geometry. These findings lay the basis for a predictive stochastic approach to upscale solute dispersion in complex porous media from the pore to the Darcy scale.