A friend of mine was experiencing multiple-second launches for Terminal. When he looked, he saw that it was occupying about 1 GB of memory.
Turns out the option to maintain ALL HISTORY EVER as scrollback was turned on. So he had the last year of compiler output and whatnot stored in a file somewhere, getting loaded into memory the first time he launched Terminal after a boot, and then eating a big chunk of his laptop's memory. Turning off the option was a major improvement.
Holy crap.
I leave Scrollback limited to "available memory", which I assume is what you're referring to, but I also compulsively hit ⌘K to clear my scrollback.
It might be parsing through logs to find out last login information.
So, like any unix os, ensure you do this:
touch $HOME/.hushlogin
That will short circuit that stuff. Like you said I've not had issues with terminals opening fast but it can't hurt. I don't give a rats when I last logged in on any unix os I log into.
Terminal has started instantaneously on all my Macs for years, since the advent of SSDs. A tiny fraction of a second.
If it is taking several seconds to launch, I think something must have gone haywire somewhere.