Book Chapter on Shiro Ikeda

EM Algorithm in Neural Network Learning

Wed, 01 Oct 2003 00:00:00 +0000

In The EM Algorithm and Related Statistical Models (STATISTICS: A Dekker series of Textbooks and Monographs 170.), Ed. by Michiko Watanabe and Kazunori Yamaguchi, Chap. 8, pp. 95–126.

ISBM: 0824747011

New York, NY/Basel: Marcel Dekker, Inc.

Authors:

Noboru Murata
Shiro Ikeda

Abstract:

(From ``introduction’’) In this article, we first review the EM algorithm from the geometrical viewpoint based on the em algorithm proposed in [7]. This geometrical concept is important to interpret various learning rules in neural networks. Then we give some examples of neural network models in which the EM algorithm implicitly appears in the learning process. From the biological point of view, it is an important problem that the EM algorithm can be appropriately implemented on real biological systems. This is usually hard, and with a special model, the Helmholtz machine, we shortly discuss the tradeoff between statistical models and biological models. In the end, we show two models of neural networks in which the EM algorithm is adopted for learning explicitly. These models are proposed mainly for practical applications, not for biological modeling, and they are applied for complicated tasks such as controlling robots.

Information Geometry and Mean Field Approximation: The $\alpha$-projection Approach

Thu, 01 Feb 2001 00:00:00 +0000

In Advanced Mean Field Methods – Theory and Practice, Ed. by Manfred Opper and David Saad, Chap. 16, pp. 241–257.

ISBN: 0262150549.

Cambridge, MA: MIT Press.

Authors:

Shun-ichi Amari
Shiro Ikeda
Hidetoshi Shimokawa

Abstract:

Information geometry is applied to mean field approximation for elucidating its properties in the spin glass model or the Boltzmann machine. The $\alpha$-divergence is used for approximation, where $\alpha$-geodesic projection plays an important role. The naive mean field approximation and TAP approximation are studied from the point of view of information geometry, which treats the intrinsic geometric structures of a family of probability distributions. The bifurcation of the $\alpha$-projection is studied, at which the uniqueness of the $\alpha$-approximation is broken.

ICA on Noisy Data: A Factor Analysis Approach

Thu, 01 Jun 2000 00:00:00 +0000

In Advances in Independent Component Analysis, Ed. by Mark Girolami, Chap. 11, pp. 201–215.

ISBN: 1852332638.

Springer-Verlag London Ltd.

Authors:

Shiro Ikeda