Neural architectures for parametric estimation of a posteriori probabilities by constrained conditional density functions

TitleNeural architectures for parametric estimation of a posteriori probabilities by constrained conditional density functions
Publication TypeConference Paper
Year of Publication1999
AuthorsArribas, J. I., J. Cid-Sueiro, T. Adali, and A. R. Figueiras-Vidal
Conference NameNeural Networks for Signal Processing - Proceedings of the IEEE Workshop
PublisherIEEE, Piscataway, NJ, United States
Conference LocationMadison, WI, USA
KeywordsAsymptotic stability, Constraint theory, Data structures, Gaussian mixture models, Joint network and data density estimation, Mathematical models, Maximum likelihood estimation, Neural networks, Probability
Abstract

A new approach to the estimation of 'a posteriori' class probabilities using neural networks, the Joint Network and Data Density Estimation (JNDDE), is presented in this paper. It is based on the estimation of the conditional data density functions, with some restrictions imposed by the classifier structure; the Bayes' rule is used to obtain the 'a posteriori' probabilities from these densities. The proposed method is applied to three different network structures: the logistic perceptron (for the binary case), the softmax perceptron (for multi-class problems) and a generalized softmax perceptron (that can be used to map arbitrarily complex probability functions). Gaussian mixture models are used for the conditional densities. The method has the advantage of establishing a distinction between the network parameters and the model parameters. Complexity on any of them can be fixed as desired. Maximum Likelihood gradient-based rules for the estimation of the parameters can be obtained. It is shown that JNDDE exhibits a more robust convergence characteristics than other methods of a posteriori probability estimation, such as those based on the minimization of a Strict Sense Bayesian (SSB) cost function.

URLhttp://www.scopus.com/inward/record.url?eid=2-s2.0-0033321049&partnerID=40&md5=7967fa377810cc0c3e6a4d9020024b80
DOI10.1109/NNSP.1999.788145