The spread of epidemics in populations is often inhomogeneous, consequently infection incidence varies between sub-populations. Age-structure is often particularly important in the dynamics of epidemics, due to the contact patterns between individuals of different ages. Public health interventions are often targeted at specific age-groups, therefore analysing the age-structure of transmission patterns is essential to evaluate the efficacy of these interventions. We develop a Bayesian model to estimate the contribution of different age-groups to the reproduction number (R) and to new infections for COVID-19 in England throughout 2021, using the ONS Infection Survey. We model a dynamic next-generation matrix in a novel way by splitting it into a static survey-derived social-contact matrix, multiplied by a low-rank dynamic matrix. We show that whilst R was typically highest for school-age children (5-11y and 12-17y) and lowest for the elderly (60y+), the former typically rose during term-time and fell during the school-holidays. The dynamics for young adults (18-29y) were particularly interesting, which increased relative to older adults in late-spring 2021 following the re-opening of entertainment venues. The R peaked for young adults in July 2021 coinciding with the period of the Euros football tournament, before rapidly dropping as the national vaccination program reached this group in August 2021. Our model is an important tool that can estimate R and attribute new infections by the infector's age, thus identifying core groups which sustain the epidemic and informing the design of targeted interventions.