Re:The Coming of Local LLMs
by whimsicalism on Tuesday April 11, 2023 @04:52PM
(#35531380)
Even with LoRA the economics are not in your favor to buy a 4090 instead of cloud training.
[ Parent
| on HN ]
Re:The Coming of Local LLMs
by thewataccount on Tuesday April 11, 2023 @04:57PM
(#35531446)
Actually yeah you're definitely right.
My perspective was for people who have other uses for them e.g. gaming or local inference. From a pure finance standpoint you're definitely right - you should rent and not buy a dedicated card. I think you'd need a few thousand hours to break even which is a few months 24/7.