Eastern subtropical China is a key region for understanding the variability of the East Asian Summer Monsoon (EASM). Multidisciplinary studies in southeastern China have shown that the summer monsoon intensity declined in the mid-late Holocene. We present a high-resolution pollen record of the last 4000 cal yr BP in Jinggang Mts of Jiangxi Province, southeastern China. The identified pollen taxa from the core can be statistically divided into three groups corresponding to evergreen, deciduous and wetland communities. The transitions between evergreen and deciduous-coniferous pollen associations is likely caused by temperature fluctuations, indicating that climate was relatively cool at 3800–3200 cal yr BP and 2200–1300 cal yr BP and warmer at 3200–2800 cal yr BP and 1300–800 cal yr BP. The vegetation study suggests that an Alnus-dominant association represents a secondary forest that usually takes place and expand after repeated forest fires. The charcoal concentration from the core depicts at least six major forest fire events since 4000 cal yr BP, most of which were followed by the development of an Alnus forest community. This result suggests that the EASM weakened toward the late Holocene and that its related decrease in moisture led to large forest wildfires. Furthermore, the rapid formation of a swamp and the subsequent development of the Alnus wetland community at ~ 550 cal yr BP suggest a gradual drying up of the lake, which was likely related to “the Little Ice Age”. As a result of a substantial burning related to an intensification of the human cultivation practices, Alnus reached its highest values in the last 200 years along with abundant wetland herbs, pioneer ferns (mainly Dicranopteris) and high charcoal concentrations. The present evidence of several sharp floristic and climate changes coincides with either the collapse or the beginning of some Chinese dynasties, which will need further research on the relationship between natural and human cultural changes.