synthetic-echo

Of course, but the dataset was mostly strings that needed to be cross-referenced with GIS data. Tried every library under the sun. The greatest speed up I got was using polars to process the mostly-string CSVs, but didn't help much. With that said, I think polars was also just released when we were working with that dataset and I'm sure there's been a lot of performance improvements since then.

Just an anecdote and not necessarily generalizable, but I can at least give one example:

I'm in academia doing ML research where, for all intents and purposes, we work exclusively in Python. We had a massive CSV dataset which required sorting, filtering, and other data transformations. Without getting into details, we had to rerun the entire process when new data came in roughly every week. Even using every trick to speed up the Python code, it took around 3 days.

I got so annoyed by it that I decided to rewrite it in a compiled language. Since it had been a few years since I've written any C/C++, which was only for a single class in undergrad and I remember very little of, I decided to give Go a try.

I was able to learn enough of the language and write up a simple program to do the data processing in less than a few hours, which reduced the time it took from 3+ days to less than 2 hours.

I unfortunately haven't had a chance or a need to write any more Go since then. I'm sure other compiled, GC languages (e.g., Nim) would've been just as productive or performant, but I know that C/C++ would've taken me much longer to figure out and would've been much harder to read/understand for the others that work with me who pretty much only know Python. I'm fairly certain that if any of them needed to add to the program, they'd be able to do so without wasting more than a day to do so.

I have been reviewing your comment history here, including the pastebin texts you have linked.

I find your perspective intriguing. Mind sharing the offline writings you've referred to, if there are additional ones to the three pastebin texts I've come across?

You mean the data that was already public and debunking half the shit he's said? You can't mean the data he refuses to actually release, do you?

So with that in mind, let me rephrase your comment.

> It's discouraging to see people that read hackernews, act in this way, and not look at the facts, and actual data, but rather use up all their energy on just blindly praising Elon.

Is he rooting out corruption and wasteful spending though? Or is he himself corrupt and attempting to stomp out agencies for his own benefit? It seems like the latter to me, considering which agencies he's targeting, half (or more) of his statements about the spending being falsehoods, the lack of proper processes, the people he's hired, etc.

I don't understand how people can be against due process, accountability, and genuine public service. Or are you just brainwashed by right-wing media?

I think they were talking about the "model giants" like OpenAI you mentioned. Not saying they're correct, but I will concede the amount of copyrighted information someone like OpenAI would want is probably (at least) an order of magnitude more than this particular case.

Very rare to see Vintage Story mentioned in the wild. It's tied with Project Zomboid for my favorite survival game. I wish more people knew about it.

They said comparatively simple. As in, simple compared to AT