《Variational Learning for Switching State-Space ModelS》.pdf

Variational Learning for Switching State-Space Mo dels Zoubin Ghahramani Georey E. Hinton Gatsby Computational Neuroscience Unit University College London 17 Queen Square London WC1N 3AR, UK Email: zoubin@gatsby.ucl.ac.uk Submitted to Neur al Computation Abstract We intro duce a new statistical mo del for time series which iteratively segments data into regimes with approximately linear dynamics and learns the parameters of each of these linear regimes. This mo del combines and generalizes two of the most widely used sto chastic time series mo dels|hidden Markov mo dels and linear dynamical systems|and is closely related to mo dels that are widely used in the con- trol and econometrics literatures. It can also b e derived by extending the mixture of exp erts neural network (Jacobs et al., 1991) to its fully dynamical version, in which b oth exp ert and gating networks are recurrent. Inferring the p osterior probabilities of the hidden states of this mo del is computationally intractable, and therefore the exact Exp ectation Maximization (EM) algorithm cannot b e applied. How- ever, we present a variational approximation that maximizes a lower b ound on the log likeliho o d and makes use of b oth the forward{backward recursions for hidden Markov mo dels and the Kalman lter recursions for linear dynamical systems. We tested the algorithm b oth on articial data sets and on a natural data set of respiration for