I'm trying to think of an example where you would want to use a factor with levels that are timestamps rather than just a vector of timestamps. And likewise for other data types, even strings. The only case I can think of would be if you wanted to limit the possible values to specific set.

My impression is that factors in R are borderline-deprecated, especially in the tidyverse, in favor of just using the equivalent non-factor vector.