There are hints that even less than 8 bits per weight might be usable (for certain cases and on custom hw). Not sure if it's practical but it is definitely interesting.

https://arxiv.org/pdf/1502.02551.pdf

https://arxiv.org/pdf/1610.00324.pdf

I wanted to have some basic idea about hardware so I did some "research" (googling) and ended up giving a short informal talk. My slides with some links are here:

http://ml.kjx.cz/hw4nn2017.pdf