I heartily recommend this talk. It and the corresponding proof-of-concept Intel userland packet processing driver[1] went a long way for me in removing a lot of the magic from low-level network packet handling and device management in Linux.

[1] https://github.com/emmericp/ixy