azlyrics is far from what I would consider simple or clean. It took over 30s to load for me and had way too much junk on the page.
Yes, there are many factors, and I clearly don't know all of them. I definitely see the problem that Google's inline (or amp) is trying to solve, but I think encouraging the right sites to grow to the top is a much better solution.
Obviously, there's many other factors at play. SEO is a complicated world, especially in such a crowded place as song lyrics. Though I noticed Google actually has an inline box for lyrics now which avoids the issue entirely.