So if your bet is right, the Y puts make money, the X calls lose money, and overall you come out winner. You wait until google is say 487 and then buy back your shares making 10 buck per share plus the money off your Y puts minus the money from the X calls. So thats just another dot product ie. C = 5 times 10 + Y times a - X times b.
Now say the trade goes south. Then you lose on the put, lose on the short, but make money on the X calls, so the dot product looks like above but with different weights. In either case, you can only lose a fixed sum worstcase ( so you statement " there's no limit to your potential losses" is definitely false ). So maximizing the money is then a constrained linear optimization problem. In a polynomial vector space, you can find X & Y quite easily using Dantzig. ( http://en.wikipedia.org/wiki/Simplex_algorithm )
HTH.
EDIT: grammar