A safe version of the experiment would be to take a standard incandescent lightbulb and run one wire to an earth ground and one wire to one side of the plug on the generator. My guess is still (although with more doubt based on how many intelligent people here have opined otherwise) that because of the enormous capacitance of the earth, the light bulb would still light up even though no clear circuit is formed.
In my quite possibly flawed mental model, electrons will be pulled and pushed from the earth across the filament, causing it to heat up. Whether this actually works (in my mental model) depends on the frequency and the capacitance of the generator, but I'm guessing that at 60Hz whatever is inside the generator is enough to get an effect. Unless someone has already done this, I guess I'll have to get a generator and try it.
Your statement implies that you are viewing the earth as a huge, perfect, capacitor. And by "perfect" I mean no series resistance.
In reality, the earth is a relatively poor electrical conductor (i.e., a high resistance [1]) and so where your mental model breaks is omitting that resistance from consideration in your model.
I.e., the equivalent circuit your 'experiment' produces would be:
G-----bulb------R--C
Theoretically, no. Your generator isn't going to pick up large static voltage from the atmosphere the way a power grid does, so it's theoretically safe to leave it ungrounded.
However, I would strongly advise against actually doing this experiment unless you are absolutely certain that your generator is isolated from the ground, and even then I would advise against it. Some risks are not worth taking even in the name of science.