Crustal pore pressure, which controls the activities of earthquakes and volcanoes, varies in response to rainfall. The status of pore pressure can be inferred from observed changes in seismic velocity. This study investigated the response of crustal pore pressure to rainfall in southwestern Japan based on time series of seismic velocity derived from ambient noise seismic interferometry. Considering the area heterogeneity, rainfall and seismic velocityobtained at each location were directly compared.We used a band-pass filter to distinguish the rainfall variability from sea level and atmospheric pressure, and then calculated the cross-correlation between rainfall and variations in S-wave velocity (Vs).A strong and mostly negative correlation between rainfall and Vs changes indicates variations in pore pressure (affecting Vs) in the deep formation due to groundwater level fluctuation. The results differ in each location, where most of the observation stations with clear negative cross-correlations were located in areas of granite. On the other hand, we could not observe clear correlations in steep mountain area, maybe because water flows through river without percolation.This finding suggests that geographical features contribute to the effect of rainfall in deep formation pore pressure. The time lag between rainfall and Vs variation constrains the permeability of the near-surface lithology, and hence the mechanism of infiltration, through the relationship of permeability to percolation rate. These analyses explain how fluctuations in the water table cause variations in pore pressure beneath confined formations with low permeability. In areas with high permeability, water percolates deeper and with longer time lags, demonstrating that lithology contributes to pore pressure changes associated with rainfall. Bylinking the variations in seismic velocity and crustal pore pressure spatially, this study shows that seismic monitoring may be useful in evaluating earthquake triggering processes or volcanic activity.