I couldn't find the origin of the permitting system in a couple minutes of googling, but I did find that there was a medallion system at least as early as 1950. It therefore seems unlikely that many of the people who currently live in San Francisco voted to impose one. It does not appear that there has been any serious effort to eliminate the system.