Body mass index (BMI) changes throughout life with age-varying genetic contributions. We investigated the genetic contribution to BMI across early life using repeated measures from the Avon Longitudinal Study of Parents and Children (ALSPAC) cohort. Random regression modelling was used to estimate the genetic covariance matrix (Kg) of BMI trajectories from ages one to 18 years with 65,930 repeated BMI measurements from 6,291 genotyped ALSPAC participants. The Kg matrix was used to estimate SNP-based heritability (\(\:{h}_{SNP}^{2}\)) from 1–18 years and genetic correlations across early life. Eigenvalue decomposition of Kg further identified age-varying genetic patterns of BMI. Finally, we investigated the impact of a polygenic score of adult BMI on the estimated genetic components across early life. The \(\:{h}_{SNP}^{2}\) was relatively constant across early life, between 23–30%. The genetic contribution to BMI in early childhood is different to that in later childhood, indicated by the diminishing genetic correlation across different ages. The eigenvalue decomposition revealed that the primary axis of variation (explaining 89% of the genetic variance in Kg) increases with age from zero and reaches a plateau in adolescence, while the second eigenfunction (explaining around 9% of Kg) represents factors with opposing effects on BMI between early and later ages. Adjusting for the adult BMI polygenic score attenuated the \(\:{h}_{SNP}^{2}\) from late childhood; for example, \(\:{h}_{SNP}^{2}\) is 29.8% (SE = 6.5%) at 18 years of age and attenuates to 14.5% (SE = 6.3%) after the adjustment. Our findings indicate that there is a different genetic profile operating during infancy compared to later childhood and adolescence.