0.25
0.5
0.75
1.25
1.5
1.75
2
Triple jump acceleration for the EM algorithm and its extrapolation-based variants
Published on Feb 25, 20073700 Views
The Aitken's acceleration is one of the most commonly used method to speed up the fixed-point iteration computation, including the EM algorithm. However, it requires to compute or approximate the Jaco
Chapter list
Triple Jump Acceleration for the EM Algorithm and Its Extrapolation-based Variants00:00
Motivation00:32
Brief Summary of Our Work01:29
Parameter Estimation Problem02:18
The EM Algorithm02:54
Taylor Expansion of M03:53
Eigenvaluesof J05:09
Convergence Rate of EM05:47
Parameterized EM (pEM)06:24
Convergence Rate of pEM(1)07:18
Convergence Rate of pEM(2)08:18
Adaptive OverrelaxedEM (aEM)08:51
Aitken’sAcceleration for EM (1)09:33
Aitken’sAcceleration for EM (2)10:52
Our Solution to Accelerate EM11:55
Triple Jump Framework (1)12:20
Triple Jump Framework (2)13:00
Triple Jump Framework (3)14:01
Triple Jump Framework (4)14:34
Advantages of TJ Framework14:52
TJEM Extrapolation (1)16:09
TJEM Extrapolation (2)16:59
TJEM Algorithm17:20
TJpEMExtrapolation17:36
TJpEMAlgorithm18:15
Convergence Properties of TJpEM Algorithm18:29
Convergence Rates of TJEM and TJpEM19:59
Convergence Rates of TJpEM with Different Learning Rates20:12
TJ2pEM Extrapolation21:48
Comparison of TJ2pEM & TJpEM22:26
TJ2pEM Algorithm23:07
Convergence Rate of TJ2pEM23:10
Convergence Rates of TJ2pEM with Different Learning Rates23:48
TJ2aEM Algorithm24:41
Why Dynamic Learning Rates25:07
Proof Sketch25:45
Data sets26:24
TJEM Faster than EM (HMM)26:53
TJEM Faster than EM (Alarm)27:35
TJEM Faster than EM (GMM)27:40
TJEM Faster than EM (Bayesian Classifier)27:42
TJpEM with Proper Learning Rate Faster than TJEM27:43
TJpEM with Large Learning Rates Slower than TJEM28:16
TJ2pEM Overcomes the Impact of Large Learning Rates28:32
aEMFaster than pEM28:43
TJ2aEM Faster than aEM(HMM)29:54
TJ2aEM Faster than aEM(Alarm)30:35
TJ2aEM Faster than aEM(GMM)30:54
TJ2aEM Faster than aEM (Bayesian Classifier)30:58
ComponentwiseTJEM31:01
Case Study: Bayesian Classifier32:18
Missing Rate = 50%32:58
Missing Rate = 90%33:20
Summary33:45
Thank You34:31