also, you're only talking about the most surface aspects of learning a language
in general pretty much every natural spoken language is equally difficult to learn, because if it's easier than what an average person can learn in their first five years of life, its speakers will invent new complexities that allow them to communicate more eloquently and efficiently. sometimes those complexities are very obvious like a larger inventory of phonemes (french has a lot of vowels, spanish doesn't), sometimes they're slightly more subtle (larger vocabularies, less regular inflection, phrasal verbs, higher segmental rate), and sometimes they're very subtle indeed (a larger inventory of idiomatic expressions, a greater cultural expectation that you will recognize literary allusions)
but about the same amount of complexity is are always there. conlangs and pidgins can avoid them (until they become native languages, as swahili and indonesian are, and i know someone who was raised speaking esperanto) and written languages aren't constrained by them because they don't rely on native language acquisition mechanisms
However spanish people suffer much more than us when learning french because we don't pronounce words the way we write them in a reliable way and we have so many sounds that are so close to them spanish speaking people don't see the difference.