You can still beat Go using "simpler" math. If you have enough compute power, you can always just minimax the whole game tree. Neural networks aren't un-mathematical; they're just a slightly more complicated technique for discovering an approximation to a function that does what you want (even if you're not sure what that function looks like internally).