News for hackers. Stuff that matters.
Re:Show HN: A radically more scalable alternative to Bayesia... by tziki on Friday January 26, 2018 @06:31PM (#16243763)
The algorithm looks nice. However, the comparison to Bayesian optimization doesn't really hold as-is. The slowness of naive O(n^3) Bayesian optimization is well-known, and while I've never personally needed a faster solution, there's literature on eg. using approximate matrix inverse or special factorizations to speed up the process. You'll be hard pressed to find anyone actually suffering from the O(n^3) limitation.
Re:Show HN: A radically more scalable alternative to Bayesia... by chroem- on Friday January 26, 2018 @07:13PM (#16243992)
Hi there,

That's a fair point about the time complexity, though the algorithm does at least demonstrate a real-world speedup of several orders of magnitude against fmfn's Bayesian Optimization implementation. However, I am not sure which performance optimizations he has implemented. Simple boils down to a handful of constant size matrix multiplies and a heap insertion at each iteration, so its performance is hard to beat.

I will update the first paragraph of the readme.

Re:Show HN: A radically more scalable alternative to Bayesia... by highd on Friday January 26, 2018 @10:46PM (#16244734)
Bayesian optimization is often applied on problems where the cost of function evaluation greatly exceeds the cost of any posterior updates etc. In this case sample efficiency is the real benchmark, and you should be looking at realistic problems with d>>2.
Re:Show HN: A radically more scalable alternative to Bayesia... by chroem- on Saturday January 27, 2018 @12:46AM (#16245032)
Hi there,

Generally speaking the heuristic for applying BO to optimization problems is that the objective function must take at least as long to evaluate as an iteration of BO. For an equally sample-efficient algorithm with negligible overhead, you can now take twice as many samples and find better optima for a large class of problems.

The shown examples are all 2D because plotting high dimensional space is rather difficult. Also, this is intended as an early proof of concept. A more formal description of the algorithm is coming, along with better examples.