Last modified: 2021-03-29
Abstract
The ability of neural networks to closely approximate unknown functions to any degree of desired accuracy has generated considerable demand for neural network research in many fields. The attractiveness of NN research stems from researchers' need to approximate models without having a prior knowledge about the true underlying function, so NNs are known as universal function approximator. All algorithms used to train NNs minimize a mean square error cost function, which is not robust in the presence of large noise such as outliers. Robust statistics introduced various techniques, for estimating the parameters of a parametric model which dealing with deviations from idealized assumptions. We will focus in this paper on one popular robust technique, that is called M-estimators, to minimize the influence of noise on the accuracy of training artificial feed-forward neural networks. We used M-estimators in order to make NN training more efficient and robust. We report simulation results to analyze and evaluate the electiveness of this approach.
Keywords
References
1. Hecht-Nielsen R. "Neurocomputing. Reading• MA: Addison-Wesley, 1990.
2. Rumelhart D. E., McClelland J. L., and the PDP research group. "Parallel Distributed Processing", vol I. Cambridge, MA: MIT Press, 1986.
3. Fahlman S. E. "Faster-learning variations on backpropagalion: An empirical study". In Proc. Connectionist Models Summer School, D. Touretzky, G. Hinton, and T. Sejnowski, Eds., pp. 38—51, 1988.
4. Platt J. "A resource-allocating network for function interpolation,''Neural Computa., vol. 3, pp. 213—225, 1991.
5. Moody J., Darken C. "Fast learning in networks of locally-tuned processing units," Neural Computa., vol. 1, pp. 281-284, 1989.
6. Tollenaere T. "Supersab: Fast adaptive backpropagation with good scaling properties," Neural Networks, vol. 3, pp. 561—573, 1990.
7.Rousseeuw P. J., Leroy A. M. "Robust Regression and Outlier Detection". New York: Wiley, 1987.8. Hampel F. R., Ronchetti E. M.,
8. Rousseeuw P. J., and Stahel W. A., "Robust. Statistics, The Approach Based on Influence Functions". New York: Wiley, 1986.
9. Hampel F. R., Ronchetti E. M., Rousseeuw P. J., and Stahel W. A., "Robust. Statistics, The Approach Based on Influence Functions". New York: Wiley, 1986.
10. Huber P. J. Robust Statistics. John Wiley & Sons, New York, 1981.
10. Wadsworth H. M., Jr., Ed., "Handbook of Statistical Methods for Engineers and Scientists". New York: McGraw-Hill, 1990.
11. Goodall C. "M-estimators of location: An outline of the theory". In D. Hoaglin, F. Mosteller, and J. W. Turkey, editors, Understanding Robust and Exploratory Data Analysis, pp. 339—403. New York, 1983.
12. Koenker R., Basset G. Jr. "Regression quantiles. Econometrica", 36: 33—50, 1978.
13. William J. J. Rey. "Introduction to Robust and QuasiRobust Statistical Methods". Springer, Berlin, Heidelberg, 1983.
14. Rumelhart D. E., Hinton G. E., Williams R. J. "Learning Representations by propagating Errors. Nature", 323, October, 9, (533-546), 1986.
15. Zhengyou Zhang. "Parameter Estimation Techniques: A Tutorial with Application to Conic Fitting". October, 1995.
16. Liano K. "Robust error measure for supervised neural network learning with outliers," IEEE Trans. Neural Networks, vol. 7, pp. 246—250, Jan. 1996.
17. Alpha V. "TAO-robust backpropagation learning algorithm". Neural Networks, pp. 1—14, 2005.
18. Hornik, K., "Multilayer Feedforward Networks are Universal Approximators", Neural Networks, Vol. 2,pp 359-366, 1989.
19. Chuang C.-C, Su S.-F, &Hsiao C.-C. "The annealing robust bacpropagation (ARBP) learning algorithm". IEEE Transaction on Neural Networks, 11(5), 10671077, 2000.
20. Ahmed M., Farag A. "A neural approach to zoom-lens camera calibration from data with outliers. Image and Vision Computing", 20, 619—630, 2002.