It is very important for the protection and utilization of water resources in the Yellow River Basin to explore the influencing factors and improvement paths of green water use efficiency based on different regions. The dynamic evolution characteristics, regional differences and internal inefficiencies of green water use efficiency for 48 cities in the Yellow River Basin from 2008 to 2018 are analyzed, based on their ecological geographical divisions, with the use of a DEA-SBM model, GML index decomposition and kernel density estimation. We further use the panel Tobit model to analyze the external influencing factors of green water use efficiency and propose ways to improve the utilization of water resources in different regions from both internal and external perspectives. The results are as follows: (1) During the study period, the green water use efficiency fluctuated between 0.58–0.67 and showed a trend of improving in arid areas and falling in ones. (2) Exploring the sources of inefficiency from the inside reveals that labor, capital, and wastewater redundancy in the semihumid area is higher, the energy redundancy in the semiarid area is higher, and the economic output in the arid area is insufficient. (3) From the GML point of view, the absolute difference in the green water efficiency of the cities in the Yellow River Basin is expanding. In terms of EC(technical efficiency index), the technical efficiency of the semiarid area has a convergence effect. In terms of TC(technical progress index), the gap in the arid area has been widening, and technology in the semihumid and semiarid areas is converging backward. (4) There are significant differences in the external factors affecting green water use efficiency in different ecogeographical regions. This study helps the government to consider eco-geographical factors when formulating water resources-related policies, and provides a scientific reference for how to better utilize water resources in different regions of the Yellow River Basin from both internal and external aspects.