Mistral 7B, the most powerful language model for its size to date, Apache 2.0

Re:Mistral 7B, the most powerful language model for its size... by godelski on Thursday September 28, 2023 @12:12AM (#37685154)

Well the checkpoint __is__ the computational graph. The graph is also all the code. But if you want it in python... that's here[0].

Please be clear, we keep asking. What are you asking for? Datasets? Training algo? What?

Comparing it to software artifacts isn't a good comparison when any program with open source code (visible or free to use) is equivalent to what's being given. You have everything you need to use, edit, and fuck around with. You don't have the exact scheme, but I'll put it this way, if you gave me the hardware I could produce a LLM of high quality from scratch using their architecture.

[0] https://github.com/mistralai/mistral-src/blob/main/mistral/m...

[ Parent | on HN ]

Re:Mistral 7B, the most powerful language model for its size... by monocasa on Thursday September 28, 2023 @08:40PM (#37697907)

That doesn't conflict with anything I've said. Yes, the checkpoint is a

It's not what Mistral's engineers edit to create this release. Just like an ELF file is necessarily contains the code flow graph, in a way extractable by experts, but isn't open source because... it's not source.

[ Parent | on HN ]

Re:Mistral 7B, the most powerful language model for its size... by zamalek on Friday September 29, 2023 @12:15PM (#37706358)

In that analogy the source of the ELF would be hand-written assembly. There aren't many layers of abstraction above the graph when it comes to ML.

[ Parent | on HN ]