Show HN: We made our own inference engine for Apple Silicon

Re:Show HN: We made our own inference engine for Apple Silicon by nodesocket on Tuesday July 15, 2025 @03:16PM (#44574806)

I just spun up a AWS EC2 g6.xlarge instance to do some llm work. The GPU is NVIDIA L4 24GB and costs $0.8048/per hour. Starting to think about switching to an Apple mac2-m2.metal instance for $0.878/ per hour. Big question is the Mac instance only has 24GB of unified memory.

[ Parent | on HN ]

Re:Show HN: We made our own inference engine for Apple Silicon by khurs on Tuesday July 15, 2025 @06:32PM (#44576572)

Unified memory doesn't compare to a Nvidia GPU, the latter is much better.

Just depends on what performance level you need.

[ Parent | on HN ]