The relationship between sea level change and a single climate indicator has been widely discussed. Despite this, few studies focused on the relationship between monthly mean sea level (MMSL) and several key impact factors, including CO2 concentration, sea ice area, and sunspots, on various time scales. In addition, research on the independent relationship between climate factors and sea level on various time scales is lacking, especially when the dependence of climate factors on Niño 3.4 is excluded. Based on this, we use wavelet coherence (WC) and partial wavelet coherence (PWC) to establish a relationship between MMSL and its influencing factors. The WC results show that the influence of climate indices on MMSL has strong regional characteristics. Sunspots affect MMSL on a scale of more than 64 months. The influence of the sea ice area on MMSL in the northern hemisphere is opposite to that in the southern hemisphere. The PWC results show that after removing the influence of Niño 3.4, the significant coherent regions of the Pacific Decadal Oscillation (PDO), Dipole Mode Index (DMI), Atlantic Multidecadal Oscillation (AMO), and Southern Oscillation Index (SOI) decrease to varying degrees on different time scales in different regions, demonstrating the influence of Niño 3.4. Our work emphasizes the independent relationship between MMSL and its influencing factors on various time scales and the use of PWC and WC to describe this relationship. The study has important reference significance for selecting the best predictors of sea level change or climate systems.