News for hackers. Stuff that matters.

A comment on Princeton ‘AI Snake Oil’ authors say GenAI hype has ‘spiraled out of control’. Parent comment

Re:Princeton ‘AI Snake Oil’ authors say GenAI hype has ‘spir... by lumost on Wednesday August 23, 2023 @11:06PM (#37244563)
We're likely way too early to call this one. Based on current hardware trends, GPT-4 will run on your phone within 6-10 years. Right now, folks are feeling out the edges of what makes sense and what doesn't make sense at extreme R&D and opex expense. In 10 years, we'd expect the winners of today to still be winners and have great margins due to declining compute costs.

Granted, if you are spending 10x the future value of the product your are offering ... then even a 10x decline in compute costs won't get you where you need to be.

[ Parent | on HN ]
Re:Princeton ‘AI Snake Oil’ authors say GenAI hype has ‘spir... by riku_iki on Wednesday August 23, 2023 @11:14PM (#37244597)
> Based on current hardware trends, GPT-4 will run on your phone within 6-10 years

they quantized model from 16 bits to 4 bits which was low hanging fruit, and looks like they can't quantize it anymore to 2 bits..

[ Parent | on HN ]
Re:Princeton ‘AI Snake Oil’ authors say GenAI hype has ‘spir... by lumost on Wednesday August 23, 2023 @11:40PM (#37244752)
CPU/GPUs are general purpose, if enough workload demand exists specialized Transformer cores will be designed. Likewise, its not at all clear that current O(N^2) self-attention is the ideal setup for larger context lengths. All to say, I'd believe we have another 8-10x algorithmic improvement in inference costs over the next 10 years. In addition to whatever Moore's law brings.
[ Parent | on HN ]
Re:Princeton ‘AI Snake Oil’ authors say GenAI hype has ‘spir... by westurner on Wednesday August 23, 2023 @11:56PM (#37244849)
Mobile TPUs/NPUs:

Pixel 6+ phones have TPUs (in addition to CPUs and an iGPU/dGPU).

Tensor Processing Unit > Products > Google Tensor https://en.wikipedia.org/wiki/Tensor_Processing_Unit

TensorFlow lite; tflite: https://www.tensorflow.org/lite

From https://github.com/hollance/neural-engine :

> The Apple Neural Engine (or ANE) is a type of NPU, which stands for Neural Processing Unit.

From https://github.com/basicmi/AI-Chip :

> A list of ICs and IPs for AI, Machine Learning and Deep Learning

[ Parent | on HN ]