Evolution tends to keep things that increase survival chances, but it can also keep useless features. Also, when the environment changes, a previously useful feature may become useless. Evolution doesn't ensure that feature will disappear, unless it provides a major disadvantage.