News for hackers. Stuff that matters.
Re:Making AMD GPUs competitive for LLM inference by paulmd on Thursday August 10, 2023 @12:00PM (#37077876)
is 70B not commercially useful because of the model storage requirements, or total inference performance, or additional memory per session that's inferencing, or what?

is the output better such that it's desirable, or is this just a case of "too much performance hit for a marginal gain"?

There are no comments on this comment yet.