Mountainous areas host a disproportionately large fraction of Earth's biodiversity, suggesting a causal relationship between mountain building and biological diversification. Mountain clade radiations are generally associated with changes in environment, climate, and the increase in heterogeneity therein during mountain building. However, examining the causal relationship between mountain building and diversification is a complex challenge, because isolating the effects of surface uplift from other abiotic (climate) or biotic variables is not straightforward. In this study, we investigate the relative contributions of abiotic climate-driven (temperature) and geology-driven (elevation) drivers on evolutionary rates of ancient groups of organisms in the South American Andes. We present regional curves of Andean elevation based on a recent compilation of paleoelevational data back to the Late Cretaceous, and analyse the diversification history of six Andean frog and lizard families that originated equally far back in time. For two clades (Aromobatidae and Leptodactylidae), we find that they diversified most rapidly during the early phase of mountain building (Late Cretaceous-Paleogene), when the first high-elevation habitats emerged in South America. The diversification of one clade (Centrolenidae) is correlated with Cenozoic temperature variations, with higher speciation rates during warm periods. The last three clades (Dendrobatidae, Hemiphractidae and Liolaemidae) are best explained by environment-independent diversification, although for Liolaemidae, an almost equally strong positive correlation was found between speciation and Andean elevation since the late Eocene. Our findings imply that throughout the long-lived history of surface uplift in the Andes, mountain building drove the diversification of different clades at different times, while not directly affecting other clades. Our study illustrates the importance of paleogeographic reconstructions that capture the complexity and heterogeneity of mountain building in our understanding of the effects that a changing environment plays in shaping biodiversity patterns observed today.