Re:Google Vizier: A Service for Black-Box Optimization
by sevensor on Monday August 27, 2018 @01:15PM
(#17852796)
Surely that reinforces my point? That what matters isn't the stopping rule but your evaluation budget? If you can afford 100 evaluations, do 100 evaluations. Pick any stopping rule based on the objective function and I can construct you a scenario that either has you spending a long time on the wrong side of it, or stopping prematurely. Might as well have predictable behavior.
Re:Google Vizier: A Service for Black-Box Optimization
by indiesolver on Monday August 27, 2018 @12:55PM
(#17852632)
I think that in the context that he meant, 64 variables = 64 hyperparameters because it was about benchmark problems solved in black-box settings. In other words, it is not about the common misconception of comparing #hyperparameters and #variables on some ML model, where the latter can be in 10^7.
Stopping rules are also a requirement of tuning. Can't just have a while True:. If an objective function doesn't improve after 100 (or any #), there needs to be logic to stop the trials since this system is apparently serving all of Alphabet.