Am using (high quality) USB for my pi storage now. Recent firmware can boot directly, though before that I was using the sd card as boot partition only, having everything else on USB. Depending on the application, network boot (i.e. completely diskless) might also be an option - probably the most reliable one, but needs wired ethernet so not possible on the Zero.