News for hackers. Stuff that matters.
Re:Anthropic’s $5B, 4-year plan to take on OpenAI by gregbander on Tuesday April 11, 2023 @06:23PM (#35532455)
Chips have a 5-7 year lead time. Apple has been shipping neural chips for years while everyone else is still designing their v1.

Apple is ahead of the game for a change getting their chips in line as the software exits alpha and goes mainstream.

Re:Anthropic’s $5B, 4-year plan to take on OpenAI by rnk on Tuesday April 11, 2023 @06:41PM (#35532636)
But they haven't exposed them to use. They are missing a tremendous opportunity. They have that unique unified memory model on the m1/m2 arms so they have something no other consumer devices have. If they exposed their neural chips they'd solidify their lead. They could sell a lot more hardware.
Re:Anthropic’s $5B, 4-year plan to take on OpenAI by gleenn on Tuesday April 11, 2023 @06:48PM (#35532738)
They are though. Apple released a library to use Apple Silicon for training via PyTorch recently, and has libraries to leverage the NE in CoreML.
Re:Anthropic’s $5B, 4-year plan to take on OpenAI by rnk on Tuesday April 11, 2023 @07:39PM (#35533217)
Thank you! I wasn't aware of that. Let me research that. May 2022 announcement. Is this suitable for the the apps like llama.cpp since it's a Python library? It appears to be a library but they didn't document how to use the underlying hardware - but I welcome more info.
Re:Anthropic’s $5B, 4-year plan to take on OpenAI by anentropic on Wednesday April 12, 2023 @11:19AM (#35541226)
> Apple Silicon for training via PyTorch recently

This is just allowing PyTorch to make use of the Apple GPU, assuming the models you want to train aren't written with hard-coded CUDA calls (I've seen many that are like that, since for a long time that was the only game in town)

PyTorch can't use the Neural Engine at all currently

AFAIK Neural Engine is only usable for inference, and only via CoreML (coremltools in Python)