During the last few decades neural networks have been increasingly used in hydrological modelling for their fundamental property of parsimony and of universal approximation of non-linear functions. For the purpose of flash flood forecasting, feed-forward and recurrent multi-layer perceptrons appear to be efficient tools. Nevertheless, their forecasting performances are sensitive to the initialization of the network parameters. We have studied the cross-validation efficiency to select initialization providing the best forecasts in real time situation. Sensitivity to initialization of feed-forward and recurrent models is compared for one-hour lead-time forecasts. This study shows that cross-validation is unable to select the best initialization. A more robust model has been designed using the median of several models outputs; in this context, this paper analyses the design of the ensemble model for both recurrent and feed-forward models.