News for hackers. Stuff that matters.
Re:Will Supercapacitors Come to AI's Rescue? by tzs on Tuesday May 06, 2025 @06:06PM (#43910135)
> Another solution is dummy calculations, which run while there are no spikes, to smooth out demand. This makes the grid see a consistent load, but it also wastes energy doing unnecessary work.

Oh god...I can see it now. Someone will try to capitalize on the hype of LLMs and the hype of cryptocurrency and try to build a combined LLM training and cryptocurrency mining facility that that runs the mining between training spikes.

Re:Will Supercapacitors Come to AI's Rescue? by ludicity on Tuesday May 06, 2025 @06:28PM (#43910303)
Oh man, I really, really wish that you hadn't said this and also that you were wrong.
Re:Will Supercapacitors Come to AI's Rescue? by permo-w on Wednesday May 07, 2025 @06:44AM (#43914119)
you really think this isn't literally the first thing that happened the second hosting these models became commercially viable?
Re:Will Supercapacitors Come to AI's Rescue? by jgalt212 on Wednesday May 07, 2025 @07:35AM (#43914380)
There wouldn't be any crypto or LLMs if the Fed hadn't printed $7 trillion.
Re:Will Supercapacitors Come to AI's Rescue? by ijustlovemath on Tuesday May 06, 2025 @06:40PM (#43910398)
YCW27
Re:Will Supercapacitors Come to AI's Rescue? by candiddevmike on Tuesday May 06, 2025 @06:42PM (#43910417)
From the same founders who brought you (or didn't, actually) maritime fusion
Re:Will Supercapacitors Come to AI's Rescue? by FridgeSeal on Tuesday May 06, 2025 @06:47PM (#43910447)
It’s ok though because YC invests in the team, better just give them another chance!!
Re:Will Supercapacitors Come to AI's Rescue? by x-complexity on Tuesday May 06, 2025 @10:19PM (#43911623)
Dummy calculations aren't even needed, if you allow the LLMs to pre-compute on the given context before inference:

https://arxiv.org/abs/2504.13171

It should be noted that this type of inference is less useful on time-sensitive tasks, but most tasks truthfully don't require such time sensitivity (there exists slack time between when the task is given & when questions are asked).

Re:Will Supercapacitors Come to AI's Rescue? by wongarsu on Tuesday May 06, 2025 @10:38PM (#43911713)
There are already some providers offering cheap LLM services that will give you a response within 24 hours instead of within seconds. That allows them to schedule tasks during low-request hours when they have spare capacity and use better batching. For some automated tasks this is perfectly acceptable. A bit of effort to accommodate, but easy to justify when it halves your inference costs
Re:Will Supercapacitors Come to AI's Rescue? by sandis on Wednesday May 07, 2025 @07:32AM (#43914364)
Any examples of such providers?
Re:Will Supercapacitors Come to AI's Rescue? by dghlsakjg on Wednesday May 07, 2025 @08:12AM (#43914667)
Certain tasks at OpenAI when I checked a few months ago. Embedding for one.
Re:Will Supercapacitors Come to AI's Rescue? by wongarsu on Wednesday May 07, 2025 @09:02AM (#43915089)
OpenAI [1] as well as Azure OpenAI, Anthropic [2], as well as Parasail [3] for all the "open source" models. There are others that I was thinking of, but those are the first I could find without my notes. Typically the batch API is 50% cheaper than live inference

1: https://platform.openai.com/docs/guides/batch

2: https://docs.anthropic.com/en/docs/build-with-claude/batch-p...

3: https://docs.parasail.io/parasail-docs/batch/batch-quickstar...

Re:Will Supercapacitors Come to AI's Rescue? by cranberryturkey on Tuesday May 06, 2025 @11:10PM (#43911883)
Re:Will Supercapacitors Come to AI's Rescue? by DaSHacka on Wednesday May 07, 2025 @05:19AM (#43913635)
>implying its not already happening