IBM and NASA build language models to make scientific knowledge more accessible

Re:IBM and NASA build language models to make scientific kno... by occamrazor on Wednesday March 13, 2024 @06:17PM (#39698258)

Note that the model is based on RoBERTa and has only 125m parameter. It is not competing against any of the new popular models, not even small ones like Phi or GeMMa.

[ Parent | on HN ]

Re:IBM and NASA build language models to make scientific kno... by 3abiton on Wednesday March 13, 2024 @11:08PM (#39700221)

Given the current state of LLMs, I am not even sure this qualify to be called an LLM.

[ Parent | on HN ]

Re:IBM and NASA build language models to make scientific kno... by mistrial9 on Wednesday March 13, 2024 @11:27PM (#39700315)

second opinion - BERT family are transformer-based, and that is a big threshold right there.. secondly I am not sure that two one-minute comments could capture what exactly went on with fine tuning or graph-based methods of constraint or whatnot.. with respect to the fitness of the production tools for intended purposes.

[ Parent | on HN ]

Re:IBM and NASA build language models to make scientific kno... by jerpint on Thursday March 14, 2024 @07:44AM (#39702616)

It’s also not meant to be a generative model - only to be used as an encoder model (they list retrieval as a potential use case )

[ Parent | on HN ]