Re:The Multi-Armed Bandit Problem and Its Solutions (2018)
by Zbrush on Tuesday February 05, 2019 @01:54PM
(#19088458)
Sorry, but I think rather than clearing things up, your post causes more confusion. First of all, you're not giving a quick primer on the multi-armed bandit problem, but an attempt at describing of one specific algorithm for it. That distinction is important, because there are several algorithms with different trade-offs, hence the linked post. Second, your description isn't very coherent: probabilities have to add up to 1, and the probability to pick a machine in step two can't simultaneously be both 0% and 50%.
FWIW, the article already contains a very accessible description of what the problem is about: "Imagine you are in a casino facing multiple slot machines and each is configured with an unknown probability of how likely you can get a reward at one play. The question is: What is the best strategy to achieve highest long-term rewards?"
Re:The Multi-Armed Bandit Problem and Its Solutions (2018)
by tomcooks on Tuesday February 05, 2019 @03:05PM
(#19089228)
I think you're illustrating ONE way to deal with multi-armed bandits, specifically an epsilon-greedy strategy
Imagine four slot machines. Each time you play, you choose one with a weighted probability, and the weight is how many times it won in the past.
So on your first play, you randomly select a machine. Now imagine that machine wins. It now has a probability of 1 vs the other three with 0. So you play it again. Now it loses. It now has a .5 probability, and the other ones have 0. Your next play will be a 50% chance of the same machine, and 50% chance of one of the others. You keep going this way and eventually you will pick the one that wins the most often most of the time, but sometimes you'll pick one of the others, just to make sure your probabilities are still correct.
How is this useful to a programmer? It's great for A/B tests. For example you come up with four different home pages, and then define success as clicking on the signup link. At first you randomly show a homepage to each visitor, but as you progress, you'll get one homepage that converts better than the others. You'll show that one most often, but sometimes show the other ones to give them a chance to "catch up", just in case your initial traffic happened to be biased.