Re:Llama3 implemented from scratch
by aDyslecticCrow on Sunday May 19, 2024 @05:40PM
(#40409961)
The only thing that has changed since 2018 is the most popular network structure to play with. The code looks the same as always; python notebooks where someone manually calculated the size of each hard-coded layer to make it fit.
[ Parent
| on HN ]
Re:Llama3 implemented from scratch
by galaxyLogic on Sunday May 19, 2024 @07:36PM
(#40410568)
> someone manually calculated the size of each hard-coded layer
I wonder shouldn't AI be the best tool to optimize itself?
Re:Llama3 implemented from scratch
by octonion137 on Monday May 20, 2024 @12:52AM
(#40412365)
In theory yes, but unfortunately AI hasn't been invented yet
Re:Llama3 implemented from scratch
by psychoslave on Monday May 20, 2024 @09:12AM
(#40415091)
I don't know, shouldn't the AI then be trapped at evaluating all possible AI implementations? And since it will face the halting problem, it won't discriminate the very best one, though it will probably be able to return the best one given a capped amount of resources that is reachable through exhaustion in its space. It won't necessarily be better than what can be provided by human beings given an equivalent amount of resources.