Re:More Descent, Less Gradient
by sandoooo on Wednesday April 29, 2020 @03:08PM
(#23022894)
Let's say I have a hyperparameter optimization task where I have to tune a simulation to some spec by varying 2-4 input parameters, and the output is a single number. I have no analytical gradient, though it's probably OK to assume the domain is smooth. Each sim takes hours to run, the entire search could take days, and I would like something that works well in parallel so I can speed up the search. What's the state of art here? Are there anything close-to-state-of-art that's useable out of the box? I've read a few papers but they don't tend to come with software.
[ Parent
| on HN ]
I think Bayesian Optimization is what you're looking for here. There's an internal tool at google ("Vizier") for which a white paper has been published that solves this exact problem. I don't know if there are any public implementations of Vizier but you could probably reverse engineer some of it from the white paper: https://static.googleusercontent.com/media/research.google.c...