There's nothing inherently slow about selection on psychological traits. To give just one striking example, there is extremely rapid selection against intelligence in advanced societies going on right now, at a rate of about 0.25-0.3 standard deviations per century.

US: https://www.pnas.org/content/suppl/2016/05/25/1523592113.DCS... (Table S11)

Taiwan: https://www.sciencedirect.com/science/article/abs/pii/S01602...

Iceland: https://www.dropbox.com/s/9vq5t6urtu930xe/2017-kong.pdf