News for hackers. Stuff that matters.

A comment on Gaussian Processes, not quite for dummies. Parent comment

Re:Gaussian Processes, not quite for dummies by nestorD on Tuesday September 17, 2019 @09:51AM (#20995036)
The raw mathematical formula uses an inversion but what you truly need is to solve a linear system (which is why most implementations store a cholesky decomposition of the covariance matrix) which can be done without transforming the matrix into a dense matrix.
[ Parent | on HN ]
Re:Gaussian Processes, not quite for dummies by bhl on Tuesday September 17, 2019 @05:31PM (#21000285)
+1 This is how I solved for the optimal parameter: instead of inverting the matrix, which is memory-heavy and computationally expensive, I used the solve function from scipy.linalg.

I looked into numpy and scipy methods for symmetry and sparsity, but my kernel matrix is still too dense.

[ Parent | on HN ]
Re:Gaussian Processes, not quite for dummies by clarkeni on Tuesday September 17, 2019 @10:20AM (#20995405)
What would you use to solve the system? Iterative methods or gradient descent even?
[ Parent | on HN ]
Re:Gaussian Processes, not quite for dummies by plus on Tuesday September 17, 2019 @11:34AM (#20996423)
Something like MINRES or GMRES or CGSTAB. Correct me if I'm wrong, but the covariance matrix in GPR should be diagonally dominant, so the diagonal of the matrix can be used as a somewhat effective preconditioner.
[ Parent | on HN ]
Re:Gaussian Processes, not quite for dummies by nestorD on Tuesday September 17, 2019 @01:12PM (#20997558)
With most kernels the matrix is symmetric and positive-definite so I would probably use a conjugate gradient: https://en.wikipedia.org/wiki/Conjugate_gradient_method
[ Parent | on HN ]
Re:Gaussian Processes, not quite for dummies by lp251 on Tuesday September 17, 2019 @10:40AM (#20995657)
yes
[ Parent | on HN ]