Lecture 3 Chapter 7
NOTE: Next lecture Friday September 30 – 11.00 – 14.00
Lecture 3 Chapter 7
NOTE: Next lecture Friday September 30 – 11.00 – 14.00
Groups (changes possible, of course)
Xiaoling Ma (CTR) Cho Su Jin Nikolaus Schönemann (ERASMUS) Andres Alayon Glazunov (TeliaSonera) Alberto Gonzales (LCN) Pablo Soldati (RT) Anders Ekman (SIP) Otto Tronarp (FOI) Jouni Rantakokko (FOI) Björn Lindgren (FOI) Lars Pääjärvi (FOI) Patrick Svedman (SB) Niklas Jaldén (SB) Isaac Skog (SB) Thanh Tùng Kim (KT) Svante Bergman (SB) Richard Abrahamsson (UU) Agnes Runqvist (UU) Lars-Johan Brännmark (UU) Erik Gudmundsson (UU) 1
2
3
4 (5?)
Instructions
Hand in the solutions according to your group number (now!)
I will make a note in the list and sort the hand-ins (during the break)
Each group picks up a pile of problems according to the group number (at the end)
Correct the solutions within the groups. Fill in the hwYY_groupXX.xls and submit to ph@kth.se (deadline Friday 30/9).
Maximum Likelihood Estimation (MLE)
MVU estimator may not exist, or may be impossible to find
MLE the most popular approach
approximately optimal
MLE is optimal for large enough data records
unbiased (asymptotically unbiased)
achieves the CRLB (asymptotically efficient)
Gaussian pdf
closed form solution --- optimization problem
it can ”always” be found numerically
convergence – divergence
convergence to local optima
EM – expectation maximization algorithm
Rationale for MLE
since data x is observed, it must have been very likely
the value of the parameter that yields the largest probability for the observed data is probably close to the true value
choose the parameters that maximize the probability
MLE
Outline of chapter 7
Example
derive CRLB
cannot find MVU
derive MLE
performance analysis (fixed number of data)
asymptotic analysis (large number of data)
Theorem 7.1, 7.3 (as properties of MLE)
Theorem 7.2, 7.4 (invariance property)
Theorem 7.5 linear models (MLE=MVU)
Invariance property of MLE
Monte Carlo analysis
nonlinear estimators exhibit
a threshold effect Generate data
Calculate the estimate
go to 1 until break
determine the sample mean of all estimates
determine the sample variance
determine PDF using histogram
Comments