Re:Nature publishes 17 parameter fits to 20 plus data points
by nearestneighbor on Thursday February 04, 2010 @11:55PM
(#1102752)
In general, it's OK to use more parameters than data points even, if you properly use regularization (such as weight decay), for example. Other times, even significantly fewer parameters than data points can be wrong.
[ Parent
| on HN ]
Re:Nature publishes 17 parameter fits to 20 plus data points
by jey on Friday February 05, 2010 @12:03AM
(#1102762)
That sounds pretty ad-hoc. Sure, you can throw an L1 or L2 regularizer on your objective function, but it should be well-motivated.
Should probably just use Gaussian process regression if you want to do inference over the space of all[1] functions in a principled (i.e. Bayesian) manner.
1. (or the space of all polynomial functions or something. I forget)
Re:Nature publishes 17 parameter fits to 20 plus data points
by _delirium on Friday February 05, 2010 @02:40AM
(#1103028)
I don't think there's any consensus in the statistics community on which of the many ways to do inference over "nearly all" functions is the better one; nonparametric regression is basically a whole field, and pretty in flux over the past 10 years.