My solution: Accept that there will be dozens or even hundreds of buffers and use helm-for-files (which lists recent buffers first) to switch between the buffers I care about at any given time. Works really well.