Void Linux is great for minimal installs. Gentoo fits the bill nicely too. Both allow for small init systems and, at least in the case of Gentoo, multiple bootloaders and initramfs tools.

VoidPup may also pique your interest. It's a puppy linux build using void packages & the xbps package manager.

I normally use fatdog64 (slackware'ish, but built from scratch), but I gave VoidPup a spin, and quite liked it.

BTW, comes in 32 & 64 bit versions, so your use case would be covered.

N270, I use Hyperbola with CWM+UXterm+ CLI/TUI software except mupdf, sxiv and mpv. SBCL and Vis it's my "IDE". It runs pretty snappy.