Anharmonicity and disorder are ubiquitous in the physics of perovskites and give rise to a variety of unique phenomena observed in scattering and spectroscopy experiments with profound consequences for device applications. Several of these phenomena still lack interpretation from a first-principles perspective. Recently, the concept of polymorphism has been introduced in calculations of perovskite compounds, yielding substantial improvements in describing their electronic and structural properties. However, hitherto no approach is available to account for anharmonicity and polymorphism in the phonon dynamics and electron-phonon couplings. Here, we develop a unified framework for the treatment of both relying on the special displacement method. Combining the self-consistent phonon theory of Hooton and the Allen-Heine theory of electron-phonon renormalized band structures, we demonstrate our approach for temperature-dependent band gaps of oxide and halide cubic perovskites, obtaining unprecedented agreement with experiments. We also uncover that polymorphism in halide perovskites induces phonon bunching and overdamping that lead to vibrational dynamics far from the ideal phonon picture, consistent with neutron scattering measurements. Moreover, our results suggest that accurate band gap predictions require corrections arising from polymorphism, spin-orbit coupling, hybrid functionals, and electron-phonon coupling. The present method provides the basis for investigating electron-phonon couplings in strongly anharmonic materials.