News for hackers. Stuff that matters.
Re:The Matrix Calculus You Need for Deep Learning by mehrdadn on Friday June 29, 2018 @04:22AM (#17423175)
Are you familiar with what the 2nd derivative of a function is? The Hessian is just that, when you have a function with multiple inputs. What's the second derivative of f(x,y) = x/y? Well there are four of them depending on the order in which you differentiate: f_xx = 0, f_xy = -1/y^2, f_yx = -1/y^2, f_yy = 2x/y^3. You just put these in a nice matrix and call it the Hessian matrix. So for functions of N input variables, there are N^2 second-derivatives, so the Hessian is the matrix of all those second derivatives.

I'm not sure how you saw it used for fitting a logistic model, but in general knowing the concavity of a function is useful for minimizing or maximizing it (e.g. by repeatedly approximating it as a 2nd-order polynomial and minimizing that), so maybe that's what you saw.

Re:The Matrix Calculus You Need for Deep Learning by thomasahle on Friday June 29, 2018 @04:54AM (#17423285)
> I'm not sure how you saw [the Hessian] used for fitting a logistic model

Probably some 2nd derivative version of gradient descent or Newton's method.

https://en.wikipedia.org/wiki/Newton%27s_method_in_optimizat...

Re:The Matrix Calculus You Need for Deep Learning by mehrdadn on Friday June 29, 2018 @06:19AM (#17423615)
Yeah that's what I was referring to when I said "approximating it as a 2nd-order polynomial and minimizing that".