I think the most difficult aspect of learning English depends on (A) the learner and (B) the goal. If your aim is to learn the language to a sufficient degree that you can handle business customers at an upscale hotel in a non-English speaking country, you're going to have different concerns to somebody trying to emigrate to and fit in with a community in the north of England.