The original sequence of urban water demand prediction fluctuates greatly and it is difficult for the grey model with two-stage weakening buffer operator to control the intensity of sequence preprocessing. Therefore, the variable weight weakening buffer operator was introduced into the modelling process, in which the weight of the operator was determined by the qualitative analysis of the researchers, and the action intensity of the operator was hereby controlled, so is the accuracy of the model prediction. Based on the statistical data of water use in Shihezi City from 2011 to 2018, we established the conventional grey model, the grey model with second-order weakening buffer operator and the improved grey model with variable weight weakening buffer operator respectively. With the combination of index analysis method, the urban water demand of Shihezi City was predicted. The results show that the accuracy of the improved model is higher than that of the other two models, and the stability and accuracy of the improved model maintains the same level in the subsequent predictions. Furthermore, the difference of the prediction results between the improved model and the index analysis method is small. This study of water demand prediction can provide a reliable reference for the planning and dispatching of regional water resources in Shihezi City.