My experience (in the UK) is that the time is dominated by people being slow to put their bags away, or putting bags away then blocking the aisle while they take off jackets or getting bags back out to extract a book.

Also airlines tend to board from front and back simultaneously, which isn't always an optimization as people join the shortest queue regardless of where their seat is.