Linked lists have generally awful cache locality. If linked lists are not the bottleneck, it's because they already aren't being used for anything where performance matters (which is generally as it should be).