OpenLLaMA 13B Released

Re:OpenLLaMA 13B Released by ozr on Sunday June 18, 2023 @03:11PM (#36383332)

Note that this model can't really be used for most code tasks. The tokenizer removed repeating spaces, so it doesn't have a valid concept of indentation.

[ Parent | on HN ]

Re:OpenLLaMA 13B Released by mk_stjames on Sunday June 18, 2023 @03:39PM (#36383632)

I was going to try and be snarky and make a comment about the superior use of tabs, but I just ran samples into the tokenizer and it doesn't recognize tabs either. :-(

[ Parent | on HN ]

Re:OpenLLaMA 13B Released by TeMPOraL on Sunday June 18, 2023 @04:45PM (#36384273)

There's still space for snark - one might observe that the model is just, wisely, excluding Python.

[ Parent | on HN ]

Re:OpenLLaMA 13B Released by baq on Sunday June 18, 2023 @03:36PM (#36383601)

So, useless for the one thing LLMs are actually properly grounded? Sounds like a self-inflicted wound.

[ Parent | on HN ]

Re:OpenLLaMA 13B Released by laurentlb on Sunday June 18, 2023 @03:45PM (#36383715)

In most languages though, you can reformat the code and get the indentation back.

[ Parent | on HN ]

Re:OpenLLaMA 13B Released by searealist on Sunday June 18, 2023 @03:48PM (#36383751)

Seems trivial to autoformat afterwards?

[ Parent | on HN ]

Re:OpenLLaMA 13B Released by teaearlgraycold on Sunday June 18, 2023 @04:05PM (#36383904)

Except with Python lol

[ Parent | on HN ]

Re:OpenLLaMA 13B Released by 2-718-281-828 on Sunday June 18, 2023 @04:14PM (#36383997)

not every language is indentation based ...

[ Parent | on HN ]

Re:OpenLLaMA 13B Released by ozr on Sunday June 18, 2023 @07:26PM (#36385593)

Not strictly indentation, any sort of repeating spaces. That expands the blast radius of the mistake a bit.

[ Parent | on HN ]

Re:OpenLLaMA 13B Released by 2-718-281-828 on Monday June 19, 2023 @02:12AM (#36387834)

which language does syntactically use repeating spaces? i can only think of print statements where sth like that would be found.

[ Parent | on HN ]

Re:OpenLLaMA 13B Released by hungrigekatze on Sunday June 18, 2023 @04:51PM (#36384322)

For some discussion on how to have the LLaMa tokenizer (properly) handle repeating spaces, please see this discussion: https://github.com/openlm-research/open_llama/issues/40

[ Parent | on HN ]

Re:OpenLLaMA 13B Released by seydor on Monday June 19, 2023 @02:01AM (#36387781)

Uff. Did they know that when they were training it?

[ Parent | on HN ]