Ecosystem-based management of marine fisheries requires the use of simulation modelling to investigate the system-level impact of candidate fisheries management strategies. However, testing of fundamental assumptions such as system structure or process formulations is rarely done. In this study, we compare the output of three different ecosystem models (Atlantis, Ecopath with Ecosim, and OSMOSE) applied to the same ecosystem (the southern Benguela), to explore which ecosystem effects of fishing are most sensitive to model uncertainty. We subjected the models to two contrasting fishing pressure scenarios, applying high fishing pressure to either small pelagic fish or to adult hake. We compared the resulting model behaviour at a system level, and also at the level of model groups. We analysed the outputs in terms of various commonly used ecosystem indicators, and found some similarities in the overall behaviour of the models, despite major differences in model formulation and assumptions. Direction of change in system-level indicators was consistent for all models under the hake pressure scenario, although discrepancies emerged under the small-pelagic-fish scenario. Studying biomass response of individual model groups was key to understanding more integrated system-level metrics. All three models are based on existing knowledge of the system, and the convergence of model results increases confidence in the robustness of the model outputs. Points of divergence in the model results suggest important areas of future study. The use of feeding guilds to provide indicators for fish species at an aggregated level was explored, and proved to be an interesting alternative to aggregation by trophic level.