Re:Will Supercapacitors Come to AI's Rescue?
by ludicity on Tuesday May 06, 2025 @06:28PM
(#43910303)
Oh man, I really, really wish that you hadn't said this and also that you were wrong.
Re:Will Supercapacitors Come to AI's Rescue?
by permo-w on Wednesday May 07, 2025 @06:44AM
(#43914119)
you really think this isn't literally the first thing that happened the second hosting these models became commercially viable?
Re:Will Supercapacitors Come to AI's Rescue?
by jgalt212 on Wednesday May 07, 2025 @07:35AM
(#43914380)
There wouldn't be any crypto or LLMs if the Fed hadn't printed $7 trillion.
Re:Will Supercapacitors Come to AI's Rescue?
by ijustlovemath on Tuesday May 06, 2025 @06:40PM
(#43910398)
YCW27
Re:Will Supercapacitors Come to AI's Rescue?
by candiddevmike on Tuesday May 06, 2025 @06:42PM
(#43910417)
From the same founders who brought you (or didn't, actually) maritime fusion
Re:Will Supercapacitors Come to AI's Rescue?
by FridgeSeal on Tuesday May 06, 2025 @06:47PM
(#43910447)
It’s ok though because YC invests in the team, better just give them another chance!!
Re:Will Supercapacitors Come to AI's Rescue?
by x-complexity on Tuesday May 06, 2025 @10:19PM
(#43911623)
Dummy calculations aren't even needed, if you allow the LLMs to pre-compute on the given context before inference:
https://arxiv.org/abs/2504.13171
It should be noted that this type of inference is less useful on time-sensitive tasks, but most tasks truthfully don't require such time sensitivity (there exists slack time between when the task is given & when questions are asked).
Re:Will Supercapacitors Come to AI's Rescue?
by wongarsu on Tuesday May 06, 2025 @10:38PM
(#43911713)
There are already some providers offering cheap LLM services that will give you a response within 24 hours instead of within seconds. That allows them to schedule tasks during low-request hours when they have spare capacity and use better batching. For some automated tasks this is perfectly acceptable. A bit of effort to accommodate, but easy to justify when it halves your inference costs
Re:Will Supercapacitors Come to AI's Rescue?
by sandis on Wednesday May 07, 2025 @07:32AM
(#43914364)
Any examples of such providers?
Re:Will Supercapacitors Come to AI's Rescue?
by dghlsakjg on Wednesday May 07, 2025 @08:12AM
(#43914667)
Certain tasks at OpenAI when I checked a few months ago. Embedding for one.
Re:Will Supercapacitors Come to AI's Rescue?
by wongarsu on Wednesday May 07, 2025 @09:02AM
(#43915089)
OpenAI [1] as well as Azure OpenAI, Anthropic [2], as well as Parasail [3] for all the "open source" models. There are others that I was thinking of, but those are the first I could find without my notes. Typically the batch API is 50% cheaper than live inference
1: https://platform.openai.com/docs/guides/batch
2: https://docs.anthropic.com/en/docs/build-with-claude/batch-p...
3: https://docs.parasail.io/parasail-docs/batch/batch-quickstar...
Re:Will Supercapacitors Come to AI's Rescue?
by cranberryturkey on Tuesday May 06, 2025 @11:10PM
(#43911883)
Oh god...I can see it now. Someone will try to capitalize on the hype of LLMs and the hype of cryptocurrency and try to build a combined LLM training and cryptocurrency mining facility that that runs the mining between training spikes.