The impact of rapid habitat loss and fragmentation on biodiversity is a major issue. However, we still lack an integrative understanding of how these changes influence biodiversity dynamics over time. In this study, we investigate the effects of these changes in terms of both niche-based and neutral dynamics. We hypothesize that habitat loss has delayed effects on neutral immigration–extinction dynamics, while edge effects and environmental heterogeneity in habitat patches have rapid effects on niche-based dynamics. We analyzed taxonomic and functional composition of 100 tree communities in a tropical dry forest landscape of New-Caledonia subject to habitat loss and fragmentation. We designed an original, process-based simulation framework, and performed Approximate Bayesian Computation to infer the influence of niche-based and neutral processes. Then, we performed partial regressions to evaluate the relationships between inferred parameter values of communities and landscape metrics (distance to edge, patch area, and habitat amount around communities), derived from either recent or past (65 yr ago) aerial photographs, while controlling for the effect of soil and topography. We found that landscape structure influences both environmental filtering and immigration. Immigration rate was positively related to past habitat amount surrounding communities. In contrast, environmental filtering was mostly affected by present landscape structure and mainly influenced by edge vicinity and topography. Our results highlight that landscape changes have contrasting spatio-temporal influences on niche-based and neutral assembly dynamics. First, landscape-level habitat loss and community isolation reduce immigration and increase demographic stochasticity, resulting in slow decline of local species diversity and extinction debt. Second, recent edge creation affects environmental filtering, incurring rapid changes in community composition by favoring species with edge-adapted strategies. Our study brings new insights about temporal impacts of landscape changes on biodiversity dynamics. We stress that landscape history critically influences these dynamics and should be taken into account in conservation policies.