Background An adverse psychological environment (e.g. stressful events or depression) has been shown to influence basal cortisol levels and cortisol response to stress. This differs depending on the adverse stimuli, but also varies across individuals and may be influenced by genetic predisposition. An insertion/deletion polymorphism in the serotonin transporter gene (5-HTTLPR) is a strong candidate in this regard. Objective To investigate how stressful life events and depression are associated with diurnal cortisol levels in community-dwelling elderly and determine whether this varies according to genetic variability in the 5-HTTLPR. Methods This population-based study included 334 subjects aged 65 and older (mean (SD) = 76.5 (6.3)). Diurnal cortisol was measured on two separate days, under quiet (basal) and stressful conditions. The number of recent major stressful events experienced during the past year was assessed from a 12-item validated questionnaire as an index of cumulative recent stressful events. Lifetime trauma was evaluated using the validated Watson's PTSD inventory, which evaluates the most severe traumatic or frightening experience according to DSM criteria. Depression was defined as having a Mini-International Neuropsychiatric Interview (MINI) diagnosis of current major depressive disorder or high levels of depressive symptoms (Center for Epidemiologic Studies-Depression Scale ≥16). 5-HTTLPR genotyping was performed on blood samples. Results Exposure to stressful life events was associated with lower basal evening cortisol levels overall, and in the participants with the 5-HTTLPR L allele but not the SS genotype. The greatest effects (over 50% decrease, p < 0.001) were observed for the LL participants having experienced multiple recent stressful events or severe lifetime traumas. Participants with the L allele also had higher evening cortisol stress response. Conversely, depression tended to be associated with a 42% higher basal morning cortisol in the SS participants specifically, but did not modify the association between stressful events and cortisol levels. Conclusion An adverse psychological environment is associated with basal cortisol levels and cortisol stress response, but this differs according to 5-HTTLPR genotype.