The North Tanzanian Divergence (NTD, eastern branch of the East African Rift) corresponds to an early stage of continental breakup. In the southern NTD, two quaternary volcanoes of the Manyara-Balangida rift (Labait, Kwaraha) have erupted primary nephelinite lavas (Mg# = 79–57) that allow characterization of their deep mantle source and the alkaline magmas that percolated through the lithosphere during rift initiation. Nephelinites are olivine- and clinopyroxene-rich, and contain up to 4 vol% magmatic phlogopite that crystallized as a liquidus phase with olivine and clinopyroxene. The presence of hydrous mineral (phlogopite) phenocrysts in Kwaraha and Labait lavas strongly suggests that the alkaline melts were H2O-bearing at the time of phlogopite crystallization (1.57–2.12 wt% H2O in phlogopite), demonstrating that phlogopite may have influenced the partitioning of water between the silicate melt and anhydrous silicate minerals (<1 ppm wt H2O clinopyroxene, 1–6 ppm wt H2O in olivine). Geochemical modeling indicates that the nephelinite magmas resulted from a low degree of partial melting (0.2–1%) of a carbonate-rich (0.3%) garnet peridotite containing ∼2 vol% phlogopite. We estimate the depth of partial melting based on primary melt compositions and empirical relations, and suggest that melting occurred at depths of 110–130 km (4 GPa) for craton-edge lavas (Kwaraha volcano) and 150 km (5 GPa) for on-craton lavas (Labait volcano), close to or below the lithosphere-asthenosphere boundary in agreement with the presence of deep refractory mantle xenoliths in Labait lavas. The depth of melting becomes gradually deeper toward the southern NTD: highly alkaline magmas in the north (Engaruka-Natron Basin) are sourced from amphibole- and CO2-rich peridotite at 75–90 km depth, whereas magmatism in the south (south Manyara Basin) is sourced from deep phlogopite- and CO2-rich garnet-peridotite beneath the Tanzania craton (e.g., at the on-craton Labait volcano). Percolation of deep asthenospheric CO2-rich alkaline magmas during their ascent may have produced strong heterogeneities in the thick sub-continental lithospheric mantle by inducing metasomatism and phlogopite crystallization in glimmerite lithologies.