Palm devices embraced the limitations of their platform; WinCE just struggled to run well on anything.
I will also dispute your "in large numbers" assertion, although I do not have access to detailed statistics. In 2008, Gartner asserted ~1.3 billion devices sold, with most growth in emerging markets. A couple of publicly available charts on Statista suggest that slightly less than 10% (~120m) of those were smartphones (Apple sold less than 1% of all phones that year at ~11m units, a ~9x growth over its just-over 1m units in 2007).
The only "smartphone" models sold meaningfully in largish numbers before the iPhone would have been BlackBerry. Everything else was an also-ran. And even BlackBerry was an also-ran compared to people just buying phones.
Fundamentally, Windows CE was flawed beyond repair, and it should have been obvious from the very first iPaq and sibling devices. I used my Palm devices every day. I rarely turned on my iPaq (available third party software was also a problem; there was lots for Palm and comparatively nothing for WinCE).