IHLayman
The moment the model gets to be bigger than the size of any one GPU's VRAM, the higher by orders of magnitude of difficulty in the process of training that model.
Also, that's just compute. What about data? Sure cloud accepts your data cheaply, but they also charge you for egress of that data. Yes you should have your data in more than one location, but if you depend on just cloud then you need it in different AZ which costs even more money to keep in sync and available for training runs.
I think for simple workloads and renting compute for a startup, cloud definitely makes sense. But the moment you try to do some serious compute for ML workloads, good luck and hope you have deep pockets.
Paper folding is incredibly powerful.
https://www.federalreservehistory.org/essays/banking-panics-...
The video is great too; I hadn’t seen that before, and it gives some examples of what happens when that generational transfer of knowledge is not carried out, not just within a discipline but also across a civilization.
So ok, you'd think that 9300 miles is close enough (50 light-ms each way), but even that can show lag because your infrastructure needs to react to whatever you are typing (even Vim nowadays will connect to a language server to do tab completion and symbol lookups) and then send you a legitimate response.
And all of this is ignoring the most obvious arguments:
* Your local silicon is fast enough to do development and has been for 10 years now. Stop using a bloated IDE.
* If you production environment is so different than your local simulation of that environment, then you have bigger problems.
* Why rent your compute per month when you can own your compute and use it however the hell you want to use it?
Maybe you are speaking more in the abstract, but I think it is important to also donate to groups who _aren’t_ in your local topology, or else you will still have wealth/benefit disparity.
> This appears to be wrong. Population growth is inexorably declining, and with renewables charging hard energy prices are going to be declining, not increasing.
I don’t agree with all of the article’s premises but this is not wrong. Population is still increasing, even if population _growth_ (first derivative) is declining. And energy usage is skyrocketing, according to the EIA (https://www.eia.gov/energyexplained/us-energy-facts/).
…don’t you see how nihilist this sounds? And also unrealistic. You will have opinions, everyone has opinions and beliefs and feelings or else they wouldn’t be able to function on a day to day basis. There is a scale to beliefs and opinions, based on how informed those beliefs and opinions are, and how well they mesh with conditions in your everyday life. It’s important to inform your opinions and beliefs; the news, however flawed in its current state, forms at least one source of that information.
I think that taking all of this as one absolute or another is unnecessarily polarizing this debate. It is very easy to argue that them news media economy is quite poisonous, but that doesn’t mean that all of the information is useless. For example: I need to prepare for a storm that is approaching the area, I need to know if a COVID wave is nearing so I know to be more careful, I need to know if parking rules change or trash day is being moved. These immediate events affect my actions directly and so I must pay attention, sifting through the other crap as necessary.
But even beyond that, this resignation to political nihilism is societally destructive. Like it or not, we are all connected in a society. Our actions affect others. We have shared infrastructure and services we all contribute to and benefit from. Part of that responsibility is a participation in politics. Is it currently toxic and despairing? Certainly. But sticking your head in the sand isn’t an option, as we depend on cooperation in order to preserve a society on any level and that requires coordination.
Should we reduce our news consumption? Of course. But we also shouldn’t eschew all outside information and live in a self-imposed bubble. Maybe skip the entertainment section and focus on stuff that impacts you directly. If there is something that is shallowly followed by the news that sounds important, of course dig into longer-form articles until you understand it in full or discard it if it is important. Discretion matters.
If there is no reason for software developers (or other IT professionals who don’t have to have physical access to the servers to do their jobs) to be in the office, and we have been far more productive at home than we have been in an office that requires 2 hours of commute, extra expenditures for food, an increased chance of viral infection, noise cancelling headphones so you don’t have to deal with the stupid gossip a couple aisles over just to focus on the task at hand, then should we be required to come in just because others prefer it when their coworkers are there too?
Can’t we just leave it up the choice of the people who are working how they would prefer to work? Or do we have to enforce a skeuomorphic working arrangement on everyone just because our managers like it better?
Unpopular opinion: I never click on twitter links anymore. It’s almost never worth it.
IMHO, 140/280/N character limits are a way to cheapen discourse. I think there is something to be said for the “density” of text: text that offers very little to think about (less dense) is vacuous but encouraged by a character limit; yet, text that is compressed into a character limit either packs too much info into a short space that requires more discourse to properly get a thought across or elides too much from the text, making it less accurate/meaningful/important. Or worse: people chain posts into long 1/907, 2/907, 3/907… trains that should be blog posts rather than requiring some other application to string the thread together.
Of course the other reason (more central to this discussion) never to click on a twitter link is that JS and an account login is required now to read the posts past a certain point. If that makes me an old man yelling at a cloud, so be it, but aren’t there better ways to handle online public discourse without sacrificing people’s privacy and security?
The three projects chosen for the argument were started by companies, but let's dig a bit deeper. Kubernetes wouldn't be possible without container technology, enabled by modules contributed to Linux, and Linux did not start in a company, but rather by a student at the University of Helsinki. Pytorch wouldn't exist without Python, which was started at CWI (another research institute) in the Netherlands. Heck, we are having this argument over a protocol developed by CERN. The references seem to indicate a bias towards machine learning, but I can tell you that the researchers working on these tools would still be doing so at a university even if the companies weren't bankrolling the salaries, and even though they are at a company, they still act as if they are in academia.
Thus, I would claim that saying "most of the open source tools today are not coming from academia" is short-sighted at best. The revolutions are happening in academia, where it is safer to do more greenfield research, as opposed to working in a company where you have to have a shorter runway to profitability.
"The problem is not those companies, but rather the consumers do not want to pay for anything online. Hence, the need for advertising, privacy issues, etc."
IMHO, the problem exists on both sides. Yes, someone needs to pay to keep the lights on, and consumers should be expected to pay a nominal fee for some services. But they shouldn't need to pay with their privacy, or by shady manipulation, or by disingenuous EULAs. I don't know the solution, but I do know that cookie tracking isn't it, nor forcing people to watch ads with eye tracking like MoviePass is suggesting, nor surreptitiously spying on people's watching habits to subsidize TVs.
Driving around double-trailers in Indiana always made me anxious, so I had to see this unnerving statement in its reality, and I found this article [0] that talks about Australian road trains. Evidently they are used on single lane roads in the Outback, which makes sense. But, I am still wondering, as there must be some warehouse to transition to single-trailer loads or else they wouldn’t be able to safely do last-mile delivery?