“Broken Neural Scaling Laws”; New Scaling Law; SotA Extrapolation of All Things
paper: https://arxiv.org/abs/2210.14891tweet thread: https://twitter.com/ethanCaballero/status/1587502829580820481
evc123
Re:OpenAI's plans according to Sam Altman
on Sunday June 04, 2023 @03:54PM (#36188955)
> Whether scaling laws hold or not, is up for debate.
The scaling laws have broken: https://arxiv.org/abs/2210.14891
Re:Large Language Models: Scaling Laws and Emergent Properties
on Thursday April 20, 2023 @09:12PM (#35648683)
But what about "Broken Neural Scaling Laws" (https://arxiv.org/abs/2210.14891)?
Re:Ask HN: What were the papers on the list Ilya Sutskever gave John C...
on Saturday February 04, 2023 @01:39PM (#34656841)
Re:OpenAI should now change their name to ClosedAI
on Monday July 20, 2020 @04:49PM (#23901745)
What is the business model of HuggingFace?
OpenAI Retro Contest Report
What do we already know about Generalization? What should we ask next about it?
Revisiting Small Batch Training for Deep Neural Networks
Revisiting Small Batch Training for Deep NNs; batch_size less than 64 is best
Re:OpenAI Charter
on Monday April 09, 2018 @02:27PM (#16795588)
What about benefitting non-human animals? Hopefully the benefits are distributed to all creatures and not just humanity.
Prefrontal cortex as a meta-reinforcement learning system [DeepMind]
Re:The problem with Facebook
on Thursday March 22, 2018 @12:26PM (#16650088)
Nah, fchollet just doesn't want pytorch to minimize keras:
The Machine Learning Reproducibility Crisis
Re:OpenAI Supporters
on Thursday February 22, 2018 @10:36AM (#16438226)
Pan-Asian Deep Learning Conference; Kuala Lumpur, Malaysia; 2028
Re:Facebook and Microsoft introduce ecosystem for interchangeable AI f...
on Friday September 08, 2017 @12:10AM (#15197797)
ONNX plans to support TensorFlow as well:
https://github.com/onnx/onnx/issues/3
Re:Where will artificial general intelligence come from?
on Monday September 04, 2017 @07:33AM (#15167436)
What are "something(s) not on our radar"?
Re:OpenAI Baselines: ACKTR and A2C
on Saturday August 19, 2017 @02:04AM (#15051942)
The paper is the documentation.
Re:OpenAI at the Dota 2 World Championships
on Saturday August 12, 2017 @12:40AM (#14996503)
Elon posted these AI fear/regulation tweets right after he retweeted the opeanai dota 2 blog post:
PyTorch v0.2.0 released
Graphcore: How to build a processor for machine intelligence
Re:Why I’m Remaking OpenAI Universe
on Monday June 26, 2017 @02:26PM (#14639076)
Arthur Juliani is about to open source his interface for connecting ML agents to Unity3D game engine:
https://twitter.com/awjuliani/status/879142906178785281
Re:Why I’m Remaking OpenAI Universe
on Monday June 26, 2017 @12:51PM (#14638211)
recruit Alex Nichol (unixpickle).
Re:Why I’m Remaking OpenAI Universe
on Monday June 26, 2017 @12:18PM (#14637908)
Evolution has been running ~10^19 experiments in parallel for billions of years: http://reducing-suffering.org/how-many-wild-animals-are-ther...
Evolution is a slow algorithm, but it had access to an absurd amount of compute (all neuronal organic matter on Earth) and environment simulation (all of physical reality on Earth) when discovering us; so the discovery of the algorithms/architectures/principles in our heads shouldn't be viewed as trivial.
Re:Why I’m Remaking OpenAI Universe
on Monday June 26, 2017 @04:03AM (#14634914)
Why I’m Remaking OpenAI Universe
Facebook's brain-computer interface presentation
Recursive Neural Networks with PyTorch
Re:Deep Reinforcement Learning: An Overview
on Sunday February 26, 2017 @11:15PM (#13741824)
What important DRL stuff does this not mention?