That was another reason to go with the RX4/5 series... fastest route to ROI.
The only 'secret', which isn't a secret, was that I realized that since this is hardware, we know the maximum settings (hash/watt) that should be possible. Therefore, I set the cards to that best setting and then tune down from there. This is the opposite train of thinking than most other miners.
Most people think, let's start low and then tune up from there to make them 'faster'. The cards crash when they can't handle the settings, so it turns out that tuning down is a better way to tune since they stop crashing when they are stable... and thus don't need tuning from there. There are 3 different sets of 'knobs' to tweak, so I had to build an algo to adjust the knobs in the right order to tune things down. I just had the concept of 'current -> next settings'.
Temperature and power fluctuations can make the cards crash too... so by always tuning down, you're always heading towards more stability instead of instability. Since neither of those could be controlled, machines would reboot randomly all the time.
The software I built was a golang daemon that ran on each machine and watched for these crashes and modified the tuning of each card individually. The daemon is pretty cool as it is effectively a task runner. I had different tasks to configure and monitor the machines as well. The machines are all independent, idempotent and self-healing workers. Reliably distributing the software to 20k different workers, is a fun challenge. There are a ton of unit tests, so that helped a lot.
If I have the energy, I may rip out the tasks from the daemon, turn it into a library and open source that. It is kind of a fun project that could be useful for others trying to manage large scale individual workers. Tasks could easily be 'apt install' or monitoring utilities. I even bundled node_exporter into the binary, so that we could monitor the machines with prometheus.
This comment is just your continued trolling with your anti-crypto trope. Please try another tactic.
Update: I got downvoted to -4 on this one after getting several upvotes. Makes me think that articbull has multiple accounts and is just gaming the system.
You are very likely mistaking normal HN herd behaviour - plenty of people are going to think bullshit to “unused hydro”.
Also “Please don't comment about the voting on comments. It never does any good, and it makes boring reading.” and “Please don't post insinuations about astroturfing, shilling, bots, brigading, foreign agents and the like. It degrades discussion and is usually mistaken. If you're worried about abuse, email [email protected] and we'll look at the data.” - https://news.ycombinator.com/newsguidelines.html
If you assume that HN is full of good people, and respond accordingly, you will learn more and have more fun. If you assume HN is full of bad people causing trouble, why would you stick around? Good communities are built upon trusting cooperators and learning the valid signals of trust.
> plenty of people are going to think bullshit to “unused hydro”.
Maybe... but the way I went up and then suddenly went down to -4, was super odd.
> If you assume that HN is full of good people, and respond accordingly, you will learn more and have more fun..
I've been around here, with an account, since 2009. It is the only community I participate in any more. I believe that HN is full of great people and that the content is about as good as you're going to get these days. This is why accounts such as articbull are becoming boring to me. All it is is anti-crypto hatred that is often not fully factual. Equating me to a tobacco exec is just icing on the cake. Sorry for calling that out.
This person is a successful troll, and a pathetic thinker.
Transmission of power is expensive and lossy. These data centers are in a very remote location that isn't heavily populated. That's why they are there to begin with! The power would absolutely go to waste otherwise. That's also why we picked that area... power was cheap because it wasn't being used.
Our GPUs are 'new old stock'. They were already produced, sitting in warehouses not doing anything, and would have gone to waste anyway.
Update: Similar with the downvotes...
First question, where do you think I got most of the karma? Sentiment here is broadly against crypto and has been for years (although it comes and goes). The pro side is loud, but enjoys far less support than you might imagine.
> Transmission of power is expensive and lossy.
No, it isn't. Transmission of power is super efficient. The EIA says the US lost only 5% of its electricity to T&D over the 2016-2020 period. [1] Expensive it may be, but that's not a reason not to do it. A single long power line from Boston to LA would only see ~25% loss from one end to the other. The whole Ohm's law thing (P=IV), we use transformers that are 99%+ efficient to raise the voltage to ~345kV and big fat wires to reduce current and resistance and hence transmission line losses.
> That's also why we picked that area... power was cheap because it wasn't being used.
It should be used to do something productive, not generate e-waste.
> Our GPUs are 'new old stock'. They were already produced, sitting in warehouses not doing anything, and would have gone to waste anyway.
Or, prices would have gone down and more people would have bought them. I can't see 3060s getting scrapped due to lack of demand when they were going for $850 a pop.
Agreed. However, I'd hope that 'against crypto' was less of one dude ranting endlessly, which is really boring and should get down voted.
> Transmission of power is super efficient.
We're not talking about efficiency. Transmission is expensive. Which goes beyond efficiency. You have to factor in the cost of the lines themselves, which are based on distance. I've seen these costs first hand, have you?
> It should be used to do something productive, not generate e-waste.
Oh yes, please define productive for me.
> Or, prices would have gone down and more people would have bought them.
They weren't for sale to the public. Nor would they ever have been.
> I can't see 3060s getting scrapped due to lack of demand when they were going for $850 a pop.
Huh? This whole thread is about RX4xx, which is what my cards are.
It would actually be less than that using a modern HVDC transmission line. Losses for HVDC are around 3% per 1000 km of cable, and around 0.7% for the converter stations at each end. So around 15% loss for a theoretical Boston-to-LA HVDC.
When those plants shut down, the power, which was transmitted directly to the plants, had nowhere to go. Sending it somewhere else requires building high voltage transmission towers, which is very expensive to build as well as maintain.
Coinmint [1] took over the old Alcoa plant and is now pulling power from that dam. As it was explained to me by the head electrician when I visited, Coinmint is doing the dam a favor by having a constant draw of power to replace what was once the smelters. They also are paying for the electricity, which helps with maintenance of the dam, which is effectively a national resource.
This is true for nuclear and some types of fossil-fuel power plants. But for hydro, all you have to do to vary the power output is adjust the amount of water running through the turbines. Plus, hydro plants that have their own storage lakes/dams have built-in storage: any water you're not using for generation can be saved in the lake for when it's needed.
(OK, some rivers have minimum flow requirements so that fish don't die and water can be used downstream etc...)
If the card crashes due to tuning, the machine crashes, and the rest of the cards are taken offline until the machine reboots. In order to change settings, you have to reboot the entire machine too. It is a pain in the ass to find optimal tuning at scale.
There is no publicly available software to 'autotune' cards at scale. It took me a long time to first learn how to tune cards (by hand tuning thousands of cards and getting a feeling for things) and then once I figured that out, I built software to do this. It required changes in the AMD driver to get it to output in the kernel messages, which card was crashing. It required updating the mining software itself. It was a nutty insane project, but it worked well. We hit our target hashrate / power usage.