This paper studies the impact of the exploitation of groundwater on the development of agriculture in China by using the county-level panel data of Henan from 1963 to 1978. The estimation of difference-in-difference model shows that: the crop yield per mu and total crop output both witness significantly faster increase from 1960s to 1970s in regions with higher density of tube well in 1970. Besides, the construction of tube well significantly alleviates the negative effect of precipitation abnormality (drought or heavy rainfall) on crop production. The important role of groundwater on agricultural development is confirmed by using the water abundance underground as instrument variable. On the aspect of the channels of the effect of groundwater, we find the exploitation of groundwater not only raises the irrigation ratio of cultivated land, but also decreases the size of salt-affected land. Thus, the exploitation of groundwater is an important force that promote the agricultural development in China.