A couple hours in even moderately chilly water -- to include off the coast of California -- will give you hypothermia in an hour, often as little as 30 minutes. If you have to float for 2 hours without being able to get out you'll have a problem.

Rip tides pull out, so let them pull you past the break then swim side to side. You shouldn't need more than 15-20 min to get past the worst of it, and can work laterally to inbound waves.