Open Source AI Is the Path Forward

Re:Open Source AI Is the Path Forward by popcorncowboy on Tuesday July 23, 2024 @12:38PM (#41048031)

> Developers can run inference on Llama 3.1 405B on their own infra at roughly 50% the cost of using closed models like GPT-4o

Does anyone have details on exactly what this means or where/how this metric gets derived?

[ Parent | on HN ]

Re:Open Source AI Is the Path Forward by rohansood15 on Tuesday July 23, 2024 @12:44PM (#41048114)

I am guessing these are prices on services like AWS Bedrock (their post is down right now).

[ Parent | on HN ]

Re:Open Source AI Is the Path Forward by PlattypusRex on Tuesday July 23, 2024 @01:08PM (#41048417)

a big chunk of that is probably the fact that you don't need to pay someone who is trying to make a profit by running inference off-premises.

[ Parent | on HN ]