nips nips2001 nips2001-83 nips2001-83-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Shun-ichi Amari, Hyeyoung Park, Tomoko Ozeki
Abstract: Singularities are ubiquitous in the parameter space of hierarchical models such as multilayer perceptrons. At singularities, the Fisher information matrix degenerates, and the Cramer-Rao paradigm does no more hold, implying that the classical model selection theory such as AIC and MDL cannot be applied. It is important to study the relation between the generalization error and the training error at singularities. The present paper demonstrates a method of analyzing these errors both for the maximum likelihood estimator and the Bayesian predictive distribution in terms of Gaussian random fields, by using simple models. 1