Understanding selection's impact on the genome is a major theme in biology. Functionally-neutral genetic regions can be affected indirectly by natural selection, via their statistical association with genes under direct selection. The genomic extent of such indirect selection, particularly across loci not physically linked to those under direct selection, remains poorly understood, as does the time scale at which indirect selection occurs. Here we use field experiments and genomic data to show that widespread statistical associations with genes known to affect fitness in stick insects, deer mice and stickleback fish cause many genetic loci across the genome to be impacted indirectly by selection. We then show that statistical associations with other, unknown causal variants make aspects of evolution more predictable in stick insects. Thus, natural selection combines with chance genetic associations to affect genome-wide evolution across linked and unlinked loci and even in modest-sized populations.