I've got a very similar setup, except with a T430s rather than an X230 (and no new speakers, even though that's probably the thing I hate the most).
I'm probably not ready to replace it yet; it ends up being a terminal more than anything else and the performance is fine for where it's at. But... every now and then when I'm browsing, it'll slow to a crawl, and I pine for the fjords/something newer.
I've been holding back partly because it's still difficult to get more than 16GiB of RAM easily, but: is the Skylake-based Purism 13 a large or incremental step up in performance?
It's difficult to quantify, but I'd say the only thing I notice is battery life. On the x230, before the mods I've made to modernize it a bit more, the battery life was somewhere south of 4 hours.
With the Librem 13 I do get 6-10 hours depending on work load.
Mostly it's been a great experience. Like the author I also miss the trackpoint.
Between the author and I, I think we cover most of the mainline OSes you'd see on these machines (debian and arch). I'm running Manjaro (xfce) with no issues.
In total though the Librem 13 was about $300 more expensive then other similar laptops.
I believe there's a lot value in that extra cost. The privacy and open source efforts mainly. We have to support these efforts more and more if we want to protect ourselves from the larger companies that continue to mine our data.