Multilayer architectures such as those used in Bayesian belief
networks and Helmholtz machines provide a powerful framework for
representing and learning higher order statistical relations among
inputs. Because exact probability calculations with these models are
often intractable, there is much interest in finding approximate
algorithms. We present an algorithm that efficiently discovers higher
order structure using EM and Gibbs sampling. The model can be
interpreted as a stochastic recurrent network in which ambiguity in
lower-level states is resolved through feedback from higher levels.
We demonstrate the performance of the algorithm on benchmark problems.
compressed postscript (7 pages, 200kB)
Mike Lewicki's home page.