That's true, but I've noticed quite a few pieces of deep learning NLP papers take pains to use competitive baselines (eg state-of-the-art non-DL techniques, more complicated than regression) and have heard that good venues will reject your paper/talk if use poor baselines