The combined effect of topography and near-surface heterogeneities on the seismic response is hardly predictable and may lead to an aggravation of the ground motion. We apply physics-based numerical simulations of 3D seismic wave propagation to highlight these effects in the case study of Arquata del Tronto, a hamlet in the Apennines that suffered irregularly distributed damage during the 2016 seismic sequence in Central Italy. We analyze the linear visco-elastic seismic response for vertically incident plane waves in terms of spectral amplification, polarization and induced torsional motion within the frequency band 1–8 Hz over a 1 km2 square area, with spatial resolution 25 m. To discern the effects of topography from those of the sub-surface structure we iterate the numerical simulations for three different versions of the structural model: one homogeneous, one with a surficial weathering layer and a soil basin and one with a complex internal structure. The numerical results confirm the correlation between topographic curvature and amplification and support a correlation between the induced torsional motion and the topographic slope. On the other hand we find that polarization does not necessarily imply ground motion amplification. In the frequency band above 4 Hz the topography-related effects are mainly aggravated by the presence of the weathering layer, even though they do not exceed the soil-related effects in the flat-topography basin. The structure below the weathering layer plays a recognizable role in the topography-related site response only for frequencies below 4 Hz.