You're right. I always presumed that nitpicking refers to picking at loose threads in something that is knit (as in something you can't help not doing). Apparently, nitpicking refers to picking nits off someone; nits as in flea eggs.