Re:Mistral 7B Paper on ArXiv
by SimplyUnknown on Wednesday October 11, 2023 @11:38AM
(#37845890)
I mean, you can't just share the weights of the model and call it a day, right? You have to share details on what and why you are doing. You must communicate this somehow. In theory, you might be able to do this in a github readme, but a paper-style document on arxiv is nicely suited for this.
> I mean, you can't just share the weights of the model and call it a day, right?
you can't?
Re:Mistral 7B Paper on ArXiv
by SimplyUnknown on Wednesday October 11, 2023 @11:52AM
(#37846075)
Obviously you can, but in the grand scheme of things people should share more details about their method so people can improve on it in the future, no?
Re:Mistral 7B Paper on ArXiv
by selfhoster11 on Wednesday October 11, 2023 @12:56PM
(#37847006)
People release models as just the weights all the time. HuggingFace makes it pretty easy to do that.
Re:Mistral 7B Paper on ArXiv
by selfhoster11 on Wednesday October 11, 2023 @12:54PM
(#37846968)
I am very confused by this comment. There is no gatekeeping in the ML/AI community. Ideas flow freely (albeit within the confines of several major Discord servers, or so it seems). Whether the author of an idea has formal training in ML and adjacent disciplines or not, whether it's published on arXiv or not, it doesn't matter - it'll be adopted if it works and/or makes it easier for people to run their GPT waifu/ baby AGI prototype.
That said, new open foundation models sized 7B and over are still a fairly rare thing to see. If someone goes through the effort of creating one of those, and especially if it has some sort of an edge against Llama 2 7B, it's not unreasonable to expect an arXiv paper to be released about it.
Isn't token completion incapable of representing AGI? AGIs need the ability to perform internal thought and deliberation, and they probably also need to be bottom-up rather than top-down.
You can't judge whether something is AGI or not from how it works, that just leads to goalpost-moving. AGI is AGI if it can do certain things, no matter whether it's token-based or top-down or anything.
Re:Mistral 7B Paper on ArXiv
by dragonwriter on Wednesday October 11, 2023 @10:36PM
(#37852885)
> Isn't token completion incapable of representing AGI?
Given the absence of a validated model (or even usable operational definition) of general intelligence, who knows? AGI might as well be an empty marketing buzzword, it isn't something about which falsifiable fact claims can be made.
> AGIs need the ability to perform internal thought and deliberation
Systems built around LLMs can do something like this, via reflection, a technique in constructing agent-based simulations using LLMs as the engine.
Re:Mistral 7B Paper on ArXiv
by selfhoster11 on Friday October 13, 2023 @03:18AM
(#37867739)
I see what you mean. In this case, I was talking about the LLM being a component of the AGI rather than the whole AGI all by itself.
I really hate the pseudo-academic gatekeeping in the AI/ML community, Google said you have no moat, we all know you have no moat, including that degree. we can all fine tune with consumer hardware we already have or even better cheaply on readily accessible clouds for this specific purpose. why are they still doing this fake academic junk.