Do we only have that handful of examples from that period or are there other contemporaneous tombs that are not pyramids? Or smaller pyramids? The thing I always thought was so weird about the pyramids is why there are so few of them. You’d think there would be lots of smaller and medium size ones leading up to the cluster of big ones if they were in fact tombs.

There are in fact lots of pyramids and flatter proto-pyramids of various sizes! The total number is probably in the thousands, I believe.

Many have also been demolished in ancient times so the stones could be reused. For example the pyramid of Djedefre, son of Khufu who built the famous Great Pyramid. Instead of competing on size, the son made a smaller pyramid of more beautiful stone, but unfortunately the stone was too valuable to let stand and was repurposed over millennia:

https://en.wikipedia.org/wiki/Pyramid_of_Djedefre

There are over a hundred pyramids identified in Egypt of varying size and age, so depends on your definition of 'handful'. But to your question, there's the mastaba tomb starting from 3000bc or so, and it's flat toped and not a pyramid. Some scholars think the pyramid idea evolved from the mastaba (by stacking increasingly smaller mastabas on top of one another; see: Step Pyramid). There are lots of them.

One fact that blew my mind recently is that Cleopatra is closer to our present time than to the creation of the pyramids.

also note that Cleopatra was Greek, not Egyptian.

"Queen of the Ptolemaic Kingdom of Egypt from 51 to 30 BC, and its last active ruler. A member of the Ptolemaic dynasty, she was a descendant of its founder Ptolemy I Soter, a Macedonian Greek general and companion of Alexander the Great."

So, inasmuch as she thought about the pyramids, it was as a colonizer.