A child’s environment is thought to be composed of different levels that interact with individual genetic propensities, with less advantaged environments suppressing genetic effects on achievement. However, studies have not tested this theory comprehensively across multiple environmental levels. Here, we quantify the contributions of child, parent, school, neighbourhood, district, and municipality factors to achievement, and investigate interactions between polygenic scores for educational attainment (EA-PGS) and environmental levels. We link population-wide administrative data on children’s standardised test results, schools and residential identifiers to the Norwegian Mother, Father and Child Cohort Study (MoBa), which includes >23,000 genotyped parent-child trios. We test for gene-environment interactions using multilevel models with interactions between EA-PGS and random effects for school and residential environments (thus remaining agnostic to specific features of environments) and use parent EA-PGS to control for gene-environment correlation. Our within-family results suggest that students’ EA-PGS interact with schools but not residential environments (neighbourhoods, districts, and municipalities), which explain negligible variance. Students’ EA-PGS explain four times the variance in achievement in the 2.5% of schools where EA-PGS associations are strongest as in the 2.5% where effects are weakest. Contrary to theory, PGS effects are stronger in less advantaged environments (lower-performing schools). None of the school sociodemographic measures we tested could explain the interaction. Schools make a greater difference to the achievements of students with lower EA-PGS, explaining 4% of the variance for students 2SD below the mean PGS, but 2% for students 2SD above the mean. Policy to reduce social inequality in achievement in Norway should focus on tackling unequal support across schools for children with difficulties, and not on differences between residential areas.