"Feature phone" is the term for cell phones that have limited additional functionality, but aren't as sophisticated or complex as a smartphone. So yeah, it's basically referring to flip phones, Blackberries, etc

BlackBerry phones were smartphones, not feature phones. They were the smartphone to beat when Apple's iPhone and the various Android phones started coming out. They were just business oriented smartphones instead of consumer oriented smartphones (like iPhoneOS and Android devices).

The ones with the pull-out keyboards? I thought they were just well-made feature phones. I could be wrong

Feature phones are pretty much limited embedded OS phones. Rarely with app installation. And almost always camera and text messaging. Maybe web access.

Blackberry devices were more like a proper PDA with phone support.

Yeah, you are correct. Though it depends on what you mean by "app installation." Many feature phones supported Java applets. I had some third-party games on my first Motorola feature phones.

Technically they're the first smartphones. To differentiate, I usually call the two feature phones vs touch phones. The non-smartphones usually don't have internet access.