Spatially resolved vibrational mapping of nanostructures is indispensable to the development and understanding of thermal nanodevices1, modulation of thermal transport2, and novel nanostructured thermoelectric materials3–5. Through the engineering of complex structures such as alloys, nanostructures, and superlattice interfaces, one can significantly alter the propagation of phonons and suppress material thermal conductivity while maintaining electrical conductivity2. Probing local vibrations and phonon dispersions in nanostructured semiconductors informs structure-property correlations and offers insights into the design and optimization of novel thermoelectric materials. There have been no correlative experiments that spatially track the modulation of phonon properties in and around nanostructures due to spatial resolution limitations of conventional optical phonon detection techniques. Here we demonstrate two-dimensional spatial mapping of phonons in a single silicon-germanium (SiGe) quantum dot (QD) using monochromated electron energy loss spectroscopy (EELS) in the transmission electron microscope (TEM). Tracking the variation of the Si optical mode in and around the QD, we observe the nanoscale modification of the composition induced redshift. We observe nonequilibrium phonons that only exist near the interface and furthermore, develop a novel technique to differentially map phonon momenta providing direct evidence that the interplay between diffusive and specular reflection largely depends on the detailed atomistic structure --a major advancement in the field. Our work unveils the nonequilibrium phonon dynamics at nanoscale interfaces and can be used to study actual nanodevices and aid in the understanding of heat dissipation near nanoscale hotspots, which is crucial for future high-performance nanoelectronics. Our work demonstrates high spatial resolution vibrational characterization of nanostructures and interfaces that can be extended to other nanostructures and superlattice systems, in terms of composition, composition gradient, and structure driven phonon dynamics.