There's nothing to fix. People work on what they want to work on. Things that seem important to you are not important to me, and the opposite. I'm OK with that.
I'm not sure. I rarely have to do that explicitly in my head. Perhaps a model should learn to infer/guess them implicitly, from context, just like I do.
what exactly is more important?
In my opinion, having a world model (for common sense) and situational awareness (e.g. through sensor fusion, or from prior conversational history, or using some externally supplied conditioning) would be far more important.
GPT-2 does not generate meaningful sentences or only by accident
You think adding POS tags would help it generate meaningful sentences?
In my opinion, having a world model (for common sense) and situational awareness (e.g. through sensor fusion, or from prior conversational history, or using some externally supplied conditioning) would be far more important. Haha you basically want a general intelligence (AGI), I want it too! And not enough persons works on "architecting" such a thing. Opencog may interest you a lot then. But the reality is many other "simpler" tasks are needed to make this happen.
having a world model (for common sense) is an NLP task There are some interesting results https://github.com/sebastianruder/NLP-progress/blob/master/e... OpenAI does not work on this task sadly, at least for now.
You think adding POS tags would help it generate meaningful sentences? I would be clearly insufficient yet necessary. I believe they already use internally a POS tagger and a dependency parser.
Interesting. Where did you see that?
GPT-2 has not (yet) been trained to predict POS tags to my knowledge, nor BERT, or ernie 2 or xlnet has, but I think they have great potential to improve POS accuracy.