News for hackers. Stuff that matters.

A comment on DeepMind’s AlphaCode Conquers Coding, Performing as Well as Humans. Parent comment

Re:DeepMind’s AlphaCode Conquers Coding, Performing as Well ... by potatoman22 on Friday December 16, 2022 @12:44AM (#34010904)
Inference is not cheap -- SOTA LLMs require 100s of gigabytes of VRAM for inference.
[ Parent | on HN ]
Re:DeepMind’s AlphaCode Conquers Coding, Performing as Well ... by andrewjl on Sunday December 18, 2022 @10:40PM (#34048165)
Apple has made a sample project available for Stable Diffusion that can run on both macOS and iOS.[1] The sample code is here.[2]

I'll concede that Stable Diffusion may or may not meet the threshold for SOTA but still think this is indicative of inference eventually becoming supported for any compelling LLM on consumer-grade hardware. The possibilities for creative tools are just too compelling.

[1] https://machinelearning.apple.com/research/stable-diffusion-...

[2] https://github.com/apple/ml-stable-diffusion

[ Parent | on HN ]