Algorithms and Programs of Dynamic Mixture Estimation (eBook)

Unified Approach to Different Types of Components

Ivan Nagy, Evgenia Suzdaleva (Autoren)

eBook Download: PDF

2017 | 1st ed. 2017
XI, 113 Seiten
Springer International Publishing (Verlag)
978-3-319-64671-8 (ISBN)

Infos

This book provides a general theoretical background for constructing the recursive Bayesian estimation algorithms for mixture models. It collects the recursive algorithms for estimating dynamic mixtures of various distributions and brings them in the unified form, providing a scheme for constructing the estimation algorithm for a mixture of components modeled by distributions with reproducible statistics. It offers the recursive estimation of dynamic mixtures, which are free of iterative processes and close to analytical solutions as much as possible. In addition, these methods can be used online and simultaneously perform learning, which improves their efficiency during estimation. The book includes detailed program codes for solving the presented theoretical tasks. Codes are implemented in the open source platform for engineering computations. The program codes given serve to illustrate the theory and demonstrate the work of the included algorithms.

Doc. Ing. Ivan Nagy, CSc. (Ph.D.), born 1956 in Prague, Czech Republic, received his CSc. (Ph.D.) in cybernetics from UTIA, Prague in 1983. In 1980, he started working as a researcher at the Institute of Information Theory and Automation of the Czech Academy of Sciences. Since 1998, he has also been a lecturer at the Czech Technical University Faculty of Transportation Sciences in Prague.

Ing. Evgenia Suzdaleva, CSc. (Ph.D.), born 1977 in Krasnoyarsk, Russia, obtained her CSc. (Ph.D.) in 2002 in system analysis at the Siberian State Aerospace University, Krasnoyarsk, Russia. Since 2004, she has been a researcher at the Institute of Information Theory and Automation at the Czech Academy of Sciences. At the same time, she works as a lecturer at the Czech Technical University Faculty of Transportation Sciences in Prague.

Inhalt

1 Introduction 71.1 On dynamic mixtures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71.2 General conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112 Basic Models 132.1 Regression model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132.1.1 Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142.1.2 Point estimates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152.1.3 Prediction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152.2 Categorical model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162.2.1 Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172.2.2 Point estimates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172.2.3 Prediction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182.3 State-space model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182.3.1 State estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183 Statistical Analysis of Dynamic Mixtures 213.1 Dynamic mixture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213.2 Unified approach to mixture estimation . . . . . . . . . . . . . . . . . . . . . . . 223.2.1 The component part . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223.2.2 The pointer part . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233.2.3 Main subtasks of mixture estimation . . . . . . . . . . . . . . . . . . . . . 233.2.4 General algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253.3 Mixture prediction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253.3.1 Pointer prediction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263.3.2 Data prediction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 284 Dynamic Mixture Estimation 294.1 Normal regression components . . . . . . . . . . . . . . . . . . . . . . . . . . . . 294.1.1 Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 304.1.2 Simple program . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 314.1.3 Comments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 334.2 Categorical components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 344.2.1 Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 354.2.2 Simple program . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 364.2.3 Comments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 374.3 State-space components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 384.3.1 Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 394.3.2 Simple program . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 404.3.3 Comments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 425 Program Codes 435.1 Main program . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 435.1.1 Comments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 455.2 Subroutines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 475.2.1 Initialization of estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . 475.2.2 Computation of proximities . . . . . . . . . . . . . . . . . . . . . . . . . . 495.2.3 Update of component statistics . . . . . . . . . . . . . . . . . . . . . . . . 525.3 Collection of programs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 546 Experiments 556.1 Mixture with regression components . . . . . . . . . . . . . . . . . . . . . . . . . 566.1.1 Well separated components . . . . . . . . . . . . . . . . . . . . . . . . . . 576.1.2 Weak components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 646.1.3 Reduced number of components . . . . . . . . . . . . . . . . . . . . . . . 646.1.4 High dimensional output . . . . . . . . . . . . . . . . . . . . . . . . . . . . 646.2 Mixture with categorical components . . . . . . . . . . . . . . . . . . . . . . . . . 696.3 Mixture with state-space components . . . . . . . . . . . . . . . . . . . . . . . . . 756.4 Case studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 756.4.1 Static normal components . . . . . . . . . . . . . . . . . . . . . . . . . . . 796.4.2 Dynamic normal components . . . . . . . . . . . . . . . . . . . . . . . . . 807 Appendix A (supporting notions) 877.1 Useful matrix formulas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 877.2 Matrix trace . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 877.3 Dirac and Kronecker functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 887.4 Gamma and beta functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 897.5 The Bayes rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 907.6 The chain rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 917.7 The natural conditions of control . . . . . . . . . . . . . . . . . . . . . . . . . . . 917.8 Conjugate Dirichlet distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . 917.8.1 The normalization constant of Dirichlet distribution . . . . . . . . . . . . 927.8.2 Statistics update with the conjugate Dirichlet distribution . . . . . . . . . 927.8.3 The parameter point estimate of the categorical model . . . . . . . . . . . 937.8.4 Data prediction with Dirichlet distribution . . . . . . . . . . . . . . . . . 947.9 Conjugate Gauss-inverse-Wishart distribution . . . . . . . . . . . . . . . . . . . 947.9.1 Statistics update for the normal regression model . . . . . . . . . . . . . . 947.9.2 The parameter point estimate of the regression model . . . . . . . . . . . 957.9.3 The proximity evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . 968 Appendix B (supporting programs) 978.1 Simulation programs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 978.1.1 The simulation of pointer values . . . . . . . . . . . . . . . . . . . . . . . 978.1.2 The simulation of mixture with regression components . . . . . . . . . . . 988.1.3 The simulation of mixture with discrete components . . . . . . . . . . . . 998.1.4 The simulation of mixture with state-space components . . . . . . . . . . 1018.2 Supporting subroutines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1038.2.1 Scilab start settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1038.2.2 The point estimation of a normal regression model . . . . . . . . . . . . . 1038.2.3 The value of a normal multivariate distribution . . . . . . . . . . . . . . . 1048.2.4 Discrete regression vector coding . . . . . . . . . . . . . . . . . . . . . . . 1058.2.5 Kalman filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1068.2.6 Matrix upper-lower factorization . . . . . . . . . . . . . . . . . . . . . . . 1078.2.7 Transition table normalization . . . . . . . . . . . . . . . . . . . . . . . . 1088.2.8 The approximation of normal pdfs by a single pdf . . . . . . . . . . . . . 108

Produkt-Details

• Erscheint lt. Verlag: 14.8.2017
• Reihe/Serie: SpringerBriefs in Statistics
• Reihe/Serie: SpringerBriefs in Statistics
• Zusatzinfo: XI, 113 p. 27 illus. in color.
• Verlagsort: Cham
• Sprache: englisch
• Themenwelt: Mathematik / Informatik ► Informatik ► Programmiersprachen / -werkzeuge
• Themenwelt: Mathematik / Informatik ► Mathematik ► Statistik
• Themenwelt: Mathematik / Informatik ► Mathematik ► Wahrscheinlichkeit / Kombinatorik
• Schlagworte: dynamic mixtures • Markov Switching Models • mixture estimation algorithms • Mixture Models • mixture prediction • mixtures of various distributions • open source programs • Recursive Bayesian estimation
• ISBN-10: 3-319-64671-0 / 3319646710
• ISBN-13: 978-3-319-64671-8 / 9783319646718