Earth’s crust responds to perturbations from various environmental factors. To evaluate this response, seismic velocity changes offer an indirect diagnostic, especially where velocity can be monitored on an ongoing basis from ambient seismic noise. Investigating the connection between the seismic velocity changes and external perturbations could be useful for characterizing dynamic activities in the crust. The seismic velocity is known to be sensitive to variations in meteorological signals such as temperature, snow, and precipitation as well as changes in sea level. Among these perturbations, the impact of variations in sea level on velocity changes inferred from seismic interferometry of ambient noise is not well known. This study investigates the influence of the ocean in a 3-year record of ambient noise seismic velocity monitoring in the Chugoku and Shikoku regions of southwest Japan. First, we applied a bandpass filter to determine the optimal period band for discriminating among different influences on seismic velocity. Then, we applied a regression analysis between the proximity of seismic station pairs to the coast and the ocean influence, as indicated by the correlation of sea level to seismic velocity changes between pairs of stations. Our study suggests that for periods between 0.0036 to 0.01 cycle/day (100–274 days), the ocean’s influence on seismic velocity decreases with increasing distance of station pairs from the coast. The increasing sea level deforms the ocean floor, affecting the stress in the adjacent coast. The stress change induced by the ocean loading may extend at least dozens of kilometers from the coast. The correlation between sea level and inland seismic velocity changes are negative or positive. Although it is difficult to clearly interpret the correlation based on simple model, they could depend on the in situ local stress, orientation of dominant crack, and hydraulic conductivity. Our study shows that seismic monitoring may be useful for evaluating the perturbation in the crust associated with an external load.