The Pile: An 800GB Dataset of Diverse Text for Language Modeling

Re:The Pile: An 800GB Dataset of Diverse Text for Language M... by remexre on Saturday January 02, 2021 @04:15AM (#25611434)

Well, if the goal is to produce an example of how one might train a language model, I think it's fine to ignore algorithmic bias issues.

But when people are talking about making e.g. AI chatbots to help teach in classes, or customer service bots, I think it's perfectly reasonable to imagine that research effort is useful to ensure the bot isn't acting like Tay did...

(And more broadly, if you're trying to use an ML model to e.g. screen candidates for hiring, you better be damn sure it's not causing discriminatory hiring practices.)

[ Parent | on HN ]

Re:The Pile: An 800GB Dataset of Diverse Text for Language M... by lambdaphagy on Saturday January 02, 2021 @08:05PM (#25618355)

I think it's fine to build NLP models with any desired property you like, including "not leaking vulgar opinions that offend the courtly manners of your society." Makes plenty of sense.

But I do wish people would be more frank and self-aware about these purposes, rather than fig-leafing it as "ethics" or "fairness". By the time you've deliberately omitting the US Congressional Record on grounds of problematicity, it's worth asking yourself whether you're making your bot polite or doing damnatio memoriae.

[ Parent | on HN ]