Also, people are nervous that the mechanization of new human capabilities will give even more power to money and less to labor, and we will live in some distopian libertarian society. I think that is a separate and realistic fear from the grief of discovering our thoughts are not as special as we told ourselves.
For me, if I am confident that the social structure will enable fulfilling lives for people, my kids and their peers, then I am ok with exploring and understanding the possibilities of mechanized cognition.
And to those objecting to the phrase mechanized cognition, I use Claude a lot and the thing I find missing is that independence of perspective that all the people I encounter have. We wake up with a perspective and goals and preferences, a why to answer what built into the protoplasm. Some technology will rebuild that aspect of our experience in a different substrate, but it doesn’t feel like a system that can’t count but can prove Riemann is quite wired correctly for that fleshy self motivated loop.
I can't even begin to make sense of that lack of basic understanding of humanity.