News for hackers. Stuff that matters.
Re:CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster tha... by echelon on Sunday October 11, 2020 @01:01PM (#24747221)
I think there's a strong chance that Swift or Rust take a lot of Python's data science cake. Both of them are extremely fast, concrete, and have lots of investment being poured into numerics and fitting into the Python/ML ecosystem.

I don't think Julia or R are going to steal this away. In fact, I think Julia is a major turn off to engineers with some of the bizarre choices they made (eg. 1-based indexing to appease math-centric backgrounds). If you don't excite engineers, you're not going to get as many library and optimization contributions.

Julia feels like it came too late in the game and the newer entrants are making quick strides in the mathematics space.

I could be totally wrong. This is just my take, and I have no stake in this fight other than I want a good, safe, highly-compatible, and fast ecosystem to work with.

Re:CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster tha... by ddragon on Sunday October 11, 2020 @04:16PM (#24748746)
Not sure what you mean about being a turn-off to engineers. I'm an electronic engineer currently working on production data engineering pipelines (which has no relation to the engineering in my degree) and I quite like the language. 1-based index is such a small "problem" compared to actual problems you encounter when developing software that is not even worth commenting in this context.

I work on a polyglot company, and parts of the pipeline have different demands. For low latency and fast development speed we use Elixir, for high throughput we use Scala, for data analytics in batch jobs we use Python. As I see, Julia is a language that is more concise and faster than Python for the number crunching I do, and has the potential to match Scala's throughput all while having multithreading/distributed that could rival the erlang VM (not in latency, but in speed and easy to use) and Scala's Akka eventually.

So as an engineer I'm excited about the potential (a single language that you can develop as fast as python that could compete with some of the top tiers in different areas of data engineering), even if it's not yet. It needs to complete the multithreading (structured parallelism so you can safely monitor and restart threads to help with reliability and with no risk of leaking, and have some library that works at the level of abstraction as the OTP/Akka), it needs native support for the infra-structure (Kafka, Prometheus) and a solid support for the web (and both of those last two can be done right now with Julia 1.5). For now it's an amazing language for exploratory data analysis and research, but I can't wait for the moment that I can safely recommend as the backend of my company's data infra-structure.

Re:CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster tha... by benhurmarcel on Monday October 12, 2020 @09:31AM (#24754142)
I guess we're talking about a different kind of engineer, but I work mostly with mechanical/aerospace engineers and they're more comfortable with 1-based indexing like in Matlab.
Re:CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster tha... by ScottPJones on Monday October 12, 2020 @09:10PM (#24761303)
Complaining about 1-based indexing in Julia is so... 4 years old! Just use OffsetArrays, and you can use 0 based, or whatever base floats your boat (start underwater with -5, for example!)
Re:CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster tha... by dklend122 on Tuesday October 13, 2020 @06:07AM (#24764001)
Where do you you see these mythical quick strides? Do you have data to back up this claim ?