Background: Trajectories of depressive symptoms over the lifespan vary between people, but it is unclear whether these differences exhibit distinct characteristics in brain structure and function. Methods: In order to compare indices of white matter microstructure and cognitive characteristics of groups with different trajectories of depressive symptoms, we examined 774 participants of the Whitehall II Imaging Sub-study, who had completed the depressive subscale of the General Health Questionnaire up to nine times over 25 years. Twenty-seven years after the first examination, participants underwent magnetic resonance imaging to characterize white matter hyperintensities (WMH) and microstructure and completed neuropsychological tests to assess cognition. Twenty-nine years after the first examination, participants completed a further cognitive screening test. Results: Using K-means cluster modelling, we identified five trajectory groups of depressive symptoms: consistently low scorers ("low"; n=505, 62.5%), a subgroup with an early peak in depression scores ("early"; n=123, 15.9%), intermediate scorers ("middle"; n=89, 11.5%), a late symptom subgroup with an increase in symptoms towards the end of the follow-up period ("late"; n=29, 3.7%), and consistently high scorers ("high"; n=28, 3.6%). The late, but not the consistently high scorers, showed higher mean diffusivity, larger volumes of WMH and impaired executive function. In addition, the late subgroup had higher Framingham Stroke Risk scores throughout the follow-up period, indicating a higher load of vascular risk factors. Conclusions: Our findings suggest that tracking depressive symptoms in the community over time may be a useful tool to identify phenotypes that show different etiologies and cognitive and brain outcomes.