Re:Show HN: A radically more scalable alternative to Bayesia...
by highd on Friday January 26, 2018 @10:46PM
(#16244734)
Bayesian optimization is often applied on problems where the cost of function evaluation greatly exceeds the cost of any posterior updates etc. In this case sample efficiency is the real benchmark, and you should be looking at realistic problems with d>>2.
Re:Show HN: A radically more scalable alternative to Bayesia...
by chroem- on Saturday January 27, 2018 @12:46AM
(#16245032)
Hi there,
Generally speaking the heuristic for applying BO to optimization problems is that the objective function must take at least as long to evaluate as an iteration of BO. For an equally sample-efficient algorithm with negligible overhead, you can now take twice as many samples and find better optima for a large class of problems.
The shown examples are all 2D because plotting high dimensional space is rather difficult. Also, this is intended as an early proof of concept. A more formal description of the algorithm is coming, along with better examples.
That's a fair point about the time complexity, though the algorithm does at least demonstrate a real-world speedup of several orders of magnitude against fmfn's Bayesian Optimization implementation. However, I am not sure which performance optimizations he has implemented. Simple boils down to a handful of constant size matrix multiplies and a heap insertion at each iteration, so its performance is hard to beat.
I will update the first paragraph of the readme.