Selection

From Sean_Carver
Revision as of 15:51, 22 April 2009 by Carver (talk | contribs) (What is Model Selection?)
Jump to: navigation, search

First, the homework from Lab W.

What is Model Selection?

What is model selection? To answer this question let's contrast model selection with parameter estimation. In the Optimization Lab, we fit lines to data. We were doing parameter estimation, trying to estimate the parameters m and b, in the formula for a line y = m*x + b. Recall my examples: we fit a line to nearly linear data and another line to nearly parabolic data.

Fitting a line to nearly linear data. Click for full size image
Fitting a line to nearly parabolic data. Click for full size image

Clearly, in the second case, it would be more appropriate to fit a parabola to the data, rather than a line. How would we do that?

A parabola is described by the formula:  y = a x^2 + m x + b. This is a line with a new term, a*x^2. To fit a parabola, first determine the errors in the data as before:

 Error(x,y,a,m,b) = a x^2 + m x + b - y

Now define the penalties, (let's be sensible and use LS rather the UG penalties):

<math> Penalty(x,y,a,m,b) = Error(x,y,a,m,b)^2 <math>

And finally the cost function

<math> Cost(a,m,b) = \sum_{data} Penalty(x,y,a,m,b) <math>