The duration of ground motion induced by earthquakes has a significant impact on the damage evolution of dams, but there are few studies on the duration of multi-directional ground motions. In order to comprehensively investigate the impact of multi-directional ground motion duration on the cumulative damage of gravity dams, a gravity dam was taken as the research object to establish a finite element model with the consideration of plastic damage model of the concrete and the unidirectional horizontal and two-directional seismic effects. Then the integrated energy duration formula was used to study the impact of integrated energy duration on the damage accumulation effect of the dam, and the correlation between energy duration in different threshold ranges and dam damage indicators were discussed. The results show that energy duration has a significant impact on the damage failure of dams. Long-term ground motions will cause more severe damage to the dam. Vertical ground motions have a certain impact on the damage failure of the dam, which should be considered in the dynamic calculation. The energy duration in the intensity threshold range of 15%-85% has the best correlation with the damage indicators. The research results can provide some reference for the seismic design of dams.