While deep learning methods have expanded the approach to solving state estimation problems in data assimilation, a robust approach requires extending them for uncertainty quantification. In this work, we propose a method to represent location-based uncertainty through random, spatiotemporally coherent displacements in physical space, which are used to sample states in the vicinity of a given reference state. We integrate this strategy into an end-to-end variational data assimilation framework, termed 4DVarNet-LU, and compare its performance with a UNet baseline, demonstrating improved reconstruction skill through the explicit use of prior state information. Trained deterministically in output and stochastically in input, we demonstrate that at inference, we see improvement in skill when using information from initial conditions. Using observation system simulation experiments based on a single-layer quasi-geostrophic model with sparse pseudo-nadir altimetry observations, we evaluate reconstruction mean and ensemble skill. Results show that 4DVarNet-LU consistently outperforms the UNet baseline in terms of RMSE and CRPS, with increased robustness to initial condition bias and improved temporal stability. While the method doesn’t propose to account for all different uncertainty sources in data assimilation, it provides a structured way for modelling and training deep learning models for dealing with location-based uncertainty. Future work may extend this approach to account for additional sources of uncertainty and end-to-end ensemble generation.