Semi-unknown players from mid-tier clubs simply cost far less in transfer fees than established high-profile players. A classic Arsene Wenger signing was Freddie Ljungberg (https://en.wikipedia.org/wiki/Fredrik_Ljungberg). He was signed from a team called Halmstads in the Swedish league. While Halmstads were doing pretty well in the Swedish league at this time, they were unknowns on the world scene. Since Arsenal had little competition in signing him and since Ljungberg was pretty unknown they were able to pay £3m - which is not a large sum at that level level. For some context, at the time the most expensive signing in the world was Denilson who Real Betis bought for £20m. However Ljungberg went on to play 216 games for Arsenal, scoring 72 goals which is a considerable number. For comparison Denilson - who was played in roughly the same position - played 186 games for Betis and scored 13 goals. Ljungberg is generally considered to be an Arsenal legend, while Denilson is thought of as an embarrassing waste of money and gets barely a footnote on the Real Betis wikipedia page.

As for what is considered a "legend" I really don't know, football is weird like that. It could be that you achieve this status by being an aggressive no-bullshit kinda guy (Vinnie Jones at Wimbledon), a world class player who stays at the same club his entire life (Ryan Giggs @ Man Utd) or just being an insanely prolific striker who scores for fun .

On this topic though, various newspapers\magazines\sites periodically publish "worst transfers" lists which are basically filler (everyone who cares already knows who will be on the lists), but which could be surprisingly enlightening for someone who's not familiar with football's transfer scene. This one's a bit advert-y but you'll get the gist: http://bleacherreport.com/articles/1610214-13-stupidest-tran...