Identifying geochemical paleo‐proxies of vegetation type in watersheds could become a powerful tool for paleoecological studies of ecosystem dynamics, particularly when commonly used proxies, such as pollen grains, are not suitable. In order to identify new paleological proxies to distinguish ecosystem types in lake records, we investigated the differences in the sediment geochemistry of lakes surrounded by two boreal forest ecosystems dominated by the same tree species: closed‐canopy black spruce‐moss forests (MF) and open‐canopy black spruce‐lichen woodlands (LW). This study was designed as a first calibration step between terrestrial modern soils and lacustrine sediments (0–1000 cal yr BP) on six lake watersheds. In a previous study, differences in the physical and geochemical properties of forest soils had been observed between these two modern ecosystems. Here we show that the geochemical properties of the sediments varied between the six lakes studied. While we did not identify geochemical indicators that could solely distinguish both ecosystem types in modern sediments, we observed intriguing differences in concentrations of C:N ratio, carbon isotopic ratio, and aluminum oxide species, and in the stabilization of their geochemical properties with depth. The C accumulation rates at millennial scale were significantly higher in MF watersheds than in LW watersheds. We suggest that these variations could result from organic matter inflows that fluctuate depending on forest density and ground vegetation cover. Further investigations on these highlighted geochemistry markers need to be performed to confirm whether they can be used to detect shifts in vegetation conditions that have occurred in the past.