It comes from the Latin saeculum which is a period of time longer than a person's life. Rome used to hold the secular games, which was an event so big, it'd never be seen again in a person's lifetime.

https://en.wikipedia.org/wiki/Secular_Games