Understanding thermal transport across metal/semiconductor interfaces is crucial for heat dissipation of electronics The dominant heat carriers in non-metals, phonons, transport elastically across most interfaces, except for a few extreme cases where the two materials that formed the interface are highly dissimilar with a large difference in Debye temperature. In this work we show that even for two materials with similar Debye temperatures (Al/Si, Al/GaN), a substantial portion of phonons will transport inelastically across their interfaces at high temperatures, significantly enhancing interface thermal conductance. Moreover, we find that interface roughness strongly affects phonon transport process. For atomically sharp interfaces, phonons are allowed to transport inelastically and interface thermal conductance linearly increases at high temperatures. With increasing interface roughness, inelastic phonon transport rapidly diminishes. Our results provide new insights on phonon transport across interfaces and open up opportunities to engineering interface thermal conductance specifically for materials of relevance to microelectronics.