You are correct that prices of small SSD are near HDD prices, but most people would want larger drives like 12 to 16TB drives in their array, and SSDs can not compete with these on price at all.
Maybe we are in the twilight of HDDs? And SSDs will compete on price across the whole storage capacity range soon? Maybe...
That is incorrect.
RaidZ(x) vdev write speed is equal to the write speed of a single drive in the vdev.
If you want a raidZ zpool to have faster writes, you add a second vdev - that would double your write speed.
A wider vdev, on the other hand (15 or 18 drives, etc.) would not have faster write speeds.
if you needed to write a LOT (let's say something like a sustained 70MB per second all the time) but the data rate was not particularly high, you might easily wear out a RAIDZ2 composed of 2TB-4TB sized cheap SSDs and kill them in a fairly short period of time, when the same RAIDZ2 composed of 2.5" 15mm height 5TB spinning drives, or 3.5" 14-16TB drives would successfully run for many years.
also re: cheap SSDs, the quad level cell tech and write cache is a big limitation in sustained write speeds:
https://www.firstpost.com/tech/news-analysis/samsung-870-qvo...
I may be heavily influenced by my own situation, but in France, SSDs still are much more expensive than HDDs. I've just bought four IrownWolves, €100 for 4 TB. The cheapest 2 TB SSD I can remember was around €180.
My point is that given the price and the usage profile, I kind of agree with the other poster: I don't have a use for tiered ZFS and would much rather they spent the time on other things.
As it is, I'm already doing a tiered storage of sorts: data that I'm currently working with requiring fast i/o is directly attached to my computer. "Cold" or less i/o sensitive data is plenty fast on spinning drives across the network.
As a home user living in an apartment, this affords another optimization: kick the NAS out and access it over the internet, because gigabit speeds become sufficient.
You sure wouldn't want to use them in any write heavy zfs application however, because you'll kill them quick in their ultimate TB write endurance. Those are triple level and quad level cell SSDs. Double the dollar figures there for anything with a ultimate lifespan TB written figure that you would want to try in a ZFS array.
At first I thought you were talking about actual raw read/write speed and how things like ARC or write caches can actually become bottlenecks when using NVMe storage, which can easily get to 200 Gbps with a small number of devices. That's being worked on via efforts like adding Direct IO.
Instead though I think you've fallen into one of those niche holes that always takes longer to get filled because there isn't much demand. Big ZFS users simply have tons and tons of drives, and with large enough arrays to spread across even rust can do alright. They'll also have more custom direction of hot/cold to different pools entirely. Smaller scale users, depending on size vs budget, may just be using pure solid state at this point, or multiple pools with more manual management. Basic 2TB SATA SSDs are down to <$170 and 2TB NVMe drives are <$190. 10 Gbps isn't much to hit, and there are lots of ways to script things up for management. RAM for buffer is pretty cheap now too. Using a mirrored NVMe-based ZIL and bypassing cache might also get many people where they want to be on sync writes.
I can see why it'd be a nice bit of polish to have hybrid pool management built-in, but I can also see how there wouldn't be a ton of demand to implement it given the various incentives in play. Might be worth seeing if there is any open work on such a thing or at least feature request.
Also to your lower comment:
>I am using TrueNAS for my ZFS solution and it doesn't offer anything out of the box to address the issues I am bringing up.
While not directly related to precisely what you're asking for, I recently started using TrueNAS myself for the first time after over a decade of using ZFS full time and one thing that immediately surprised me is how unexpectedly limited the GUI is. TONS and tons of normal ZFS functionality is not exposed there for no reason I can understand. However, it's all still supported, it's still FreeBSD and normal ZFS underneath. You can still pull up a shell and manipulate things via the commandline, or (better for some stuff) modify the GUI framework helpers to customize things like pool creation options. The power is still there at least, though other issues like inexplicably shitty user support in the GUI show it's clearly aimed at home/SoHo, maybe with some SMB roles.