Apple: Transformer architecture optimized for Apple Silicon

Re:Apple: Transformer architecture optimized for Apple Silicon by endisneigh on Thursday March 23, 2023 @07:22PM (#35282925)

yes, there are billions of parameters necessary. but large language models only came out about 5 years ago. I'm confident 5 years from now the parameters necessary to get gpt-4 performance will be decreased orders of magnitude.

at the very least, even if that's not the case, inference will be drastically less gpu heavy by then I suspect.

[ Parent | on HN ]

Re:Apple: Transformer architecture optimized for Apple Silicon by chatmasta on Thursday March 23, 2023 @07:39PM (#35283089)

There will also be hardware improvements (as always) and ASIC chips specifically designed for running this kind of model.

[ Parent | on HN ]

Re:Apple: Transformer architecture optimized for Apple Silicon by s1k3s on Thursday March 23, 2023 @07:23PM (#35282938)

Wait, so there's a way to make a model as smart as GPT but with less parameters? Isn't that why it's so good?

[ Parent | on HN ]

Re:Apple: Transformer architecture optimized for Apple Silicon by month13 on Thursday March 23, 2023 @07:28PM (#35282996)

This is an older paper, but DeepMind alleges in their Chinchilla paper that far better performance can be extracted with fewer parameters; quote

"We find that current large language models are significantly under-trained, a consequence of the recent focus on scaling language models whilst keeping the amount of training data constant."

It's difficult to evaluate a LLM's performance as it's all qualitative, but Meta's LLaMA has been doing quite well, at even 13B parameters.

[ Parent | on HN ]

Re:Apple: Transformer architecture optimized for Apple Silicon by astrange on Thursday March 23, 2023 @10:22PM (#35284279)

Chinchilla is aimed at finding a cost-performance tradeoff as well, not the optimal amount of training. If cost is no barrier because it'll be used forever, then probably there's no amount of training that's good enough.

[ Parent | on HN ]

Re:Apple: Transformer architecture optimized for Apple Silicon by monocasa on Thursday March 23, 2023 @07:35PM (#35283055)

The rumor I've heard is that GPT4 didn't meaningfully increase the parameter count versus GPT3.5, but instead focused on training and structural improvements.

[ Parent | on HN ]

Re:Apple: Transformer architecture optimized for Apple Silicon by qumpis on Thursday March 23, 2023 @09:05PM (#35283734)

Well the inference time of gpt4 seems to be far greater than gpt3, so it could hint a difference in parameters count.

[ Parent | on HN ]

Re:Apple: Transformer architecture optimized for Apple Silicon by LesZedCB on Friday March 24, 2023 @01:15AM (#35285331)

if you watch their announcement Livestream video it looked just as fast as normal ChatGPT.

I think what we have access to is a fair bit slower.

[ Parent | on HN ]

Re:Apple: Transformer architecture optimized for Apple Silicon by endisneigh on Thursday March 23, 2023 @07:25PM (#35282956)

that's a complicated question to answer. what I'd say is that more parameters makes the model more robust, but there are diminishing returns. optimizations are under way

[ Parent | on HN ]

Re:Apple: Transformer architecture optimized for Apple Silicon by jazzyjackson on Thursday March 23, 2023 @09:02PM (#35283714)

You can train a small model to behave like the large model at a subset of tasks.

[ Parent | on HN ]

Re:Apple: Transformer architecture optimized for Apple Silicon by tlrobinson on Thursday March 23, 2023 @08:58PM (#35283689)

I could also imagine a sort of two-tier approach, where the on-device model can handle the majority of queries, but recognize when it should pass the query on to a larger model running in the cloud.

[ Parent | on HN ]

Re:Apple: Transformer architecture optimized for Apple Silicon by dachryn on Friday March 24, 2023 @06:05AM (#35287080)

dont underestimate how many of those parameters are actually necessary to support multiple languages.

If you focus on english only, this can easily reduce the paramters 5fold

[ Parent | on HN ]

Re:Apple: Transformer architecture optimized for Apple Silicon by spacebanana7 on Friday March 24, 2023 @06:44AM (#35287334)

Could you explain how supporting multiple languages increases the parameter count so much? I'm genuinely curious.

LLMs seem to be comfortable with hundreds of programming languages, DSLs and application specific syntaxes so how does supporting a couple more natural languages become so expensive?

I see how more training data would be needed, but I don't understand how that maps to a greater parameter count.

[ Parent | on HN ]