Talk:Multiple regression

This task needs more clarification, like a link to a suitable wikipedia page. —Donal Fellows 17:04, 29 June 2009 (UTC)

This task requires merging with Polynomial Fitting already representing least squares approximation example in the basis {1, x, x²}. Linear regression is just same in the basis {1, x}. --Dmitry-kazakov 18:31, 29 June 2009 (UTC)

OK, there's now implementations in two languages. It's not clear to me how this is different from the polynomial fitting task either, but I'm a completionist (for Tcl) and not a statistician... —Donal Fellows 10:34, 9 July 2009 (UTC)

An explanation from the Lapack documentation may be helpful. [1] The idea is that you want to model a set of empirical data

$\{(x_{1},F(x_{1}))\dots (x_{m},F(x_{m}))\}$

by fitting it to a function of the form

${\hat {F}}(x)=\sum _{i=1}^{n}\beta _{i}f_{i}(x)$

where you've already chosen the $f_{i}$ functions and only the $\beta$'s need to be determined. The number of data points $m$ generally will exceed the number of functions in the model, $n$, and the functions $f_{i}$ can be anything, not just $x^{i}$ as in the case of polynomial curve fitting.

I don't believe the Ruby and Tcl solutions on the page solve the general case of this problem because they assume a polynomial model. I propose that the task be clarified to stipulate that the inputs are two tables or matrices of numbers, one containing all values of $f_{j}(x_{i})$ and the other containing all values of $F(x_{i})$ with $i$ ranging from 1 to $m$ and $j$ ranging from 1 to $n$, and $m>n$. --Sluggo 12:52, 9 August 2009 (UTC)

The thing that worried me was that there wasn't any code in there to determine whether the degree of the polynomial selected was justified. I had to hard-code the depth of polynomial to try in the example code. I think the fitting engine itself doesn't care; you can present any sampled function you want for fitting. (I suppose many of the other more-advanced statistics tasks have this same problem; they require an initial “and magic happens here” to have happened before you can make use of them.) —Donal Fellows 13:47, 9 August 2009 (UTC)

You can always fit any empirical data to a polynomial. This task is about fitting it to functions of a more general form (i.e., a different basis). For example, by selecting the functions $f_{i}$ as sinusoids with appropriate frequencies, it should be possible to obtain Fourier coefficients (albeit less efficiently than with a dedicated FFT solver). I don't see how the code in the given solutions could be used to do that. --Sluggo 16:57, 9 August 2009 (UTC)