This is an extremely wrong viewpoint. "Many extremely important* papers have been published on arxiv alone - no other publisher. Just off the top of my head, without looking, ELMo from allenAI (the paper that started the contextualized representations revolution for BERT/attention) is only on arxiv.