μblog: engineering from the trenches

When trying to do parameter estimation given a set of data, there are typically two approaches: least squares estimation and maximum likelihood estimation. In both cases, a model must be constructed where the first case tries to fit the data to the model by minimizing residual errors while the second method tries to estimate the probability density function associated with the collected data and thereby determine the parameters. While trying to make sense of things, I found this tutorial on maximum likelihood estimation by In Jae Myung of Ohio State University to be very helpful as it provided a description as well as MATLAB code examples. (He also also publishes a list of books currently on his bookshelf!)

Update: Here is a link to a paper outlining R.A. Fishers arrival at the concept of maximum likelihood. An interesting thing to note here is that given a likelihood function P, log(P) is often maximized, yielding maximum likelihood because the function would have to be differentiated and given that the probability distribution of many naturally occurring events is Gaussian, differentiating the logarithm of such a probability density function just makes more sense.

del.icio.us | digg