News for hackers. Stuff that matters.
Re:We don't need data scientists, we need data engineers by proverbialbunny on Thursday January 14, 2021 @01:29PM (#25780124)
If they're using a dynamically typed language to do monetary calculations, it's not going to be ideal.

Researchers do not need to have deep programming experience, but they have to be comfortable enough to use an environment that can lend itself itself to the problem at hand. On the quant side, unlike on the data science side, the barrier of entry on the programming side is a bit higher. To solve this problem many firms have their own internal programming language.

Re:We don't need data scientists, we need data engineers by oivey on Thursday January 14, 2021 @01:49PM (#25780450)
This is dogmatism swung too far in the other direction, IMO. There are many, many successful production code bases written in dynamic languages. In my own experience as a vision scientist/engineer, there is tremendous value in being able to quickly whip up a concept in Python and then being able to easily visualize the results. Doing this exploration in C++ is wasteful. Implementation takes much longer, the correctness brought by static typing is dubious since the code isn’t in prod, and the canned CV/visualization libraries are fewer and frequently suck in at least some way. That said, there is also tremendous value in understanding how to map your Python prototype into production code, too. Someone strong in this field can do both.
Re:We don't need data scientists, we need data engineers by proverbialbunny on Thursday January 14, 2021 @02:00PM (#25780601)
This was addressed in the previous comment

>On the quant side, *unlike on the data science side*,

Vision scientist is on the data science side. You're not dealing with monetary values where floating point error compounds on itself to the point your models become garbage. Quant work is it's own unique field with its own unique prerequisites.

Re:We don't need data scientists, we need data engineers by oivey on Thursday January 14, 2021 @02:23PM (#25781002)
Nothing precludes you from doing integer arithmetic in a dynamic language.

I’m not a quant and this isn’t my area of expertise, but, for example, I’m pretty sure various differential equation solving methods depend on variables taking on continuous values, so floating point basically must be used. Understanding the impact of that is definitely very important. Analogously, I frequently run into numerical precision issues in image processing. Understanding how numbers are represented on a computer isn’t unique to being a quant. Understanding how the choice of representation can impact prod is also not unique to being a quant. The dynamicness of the language isn’t particularly relevant, either.

Re:We don't need data scientists, we need data engineers by proverbialbunny on Thursday January 14, 2021 @03:24PM (#25781987)
>Nothing precludes you from doing integer arithmetic in a dynamic language.

You would be surprised. The second you use pandas with a custom data type (let alone any other library you'd want to use) it can randomly auto convert it to a float. Furthermore identifying when it randomly converts the type on you is a pain.

>so floating point basically must be used.

Quants tend to use fixed precision types. It is like a float in every way, except base 10 instead of base 2 so there is no floating point error.

Re:We don't need data scientists, we need data engineers by dragonwriter on Monday January 18, 2021 @06:49AM (#25820789)
> The second you use pandas with a custom data type

That's a pandas (and maybe numpy) issue, not a dynamic language issue. (If you want to generalize from the specific libraries more accurately than “dynamic language”, it's “using a low-level library whose type system doesn't match the host language type system” issue.

> Quants tend to use fixed precision types. It is like a float in every way, except base 10 instead of base 2 so there is no floating point error.

No, a type that is like binary floating point in every way except base 10 instead of base 2 would be decimal floating point, not fixed point. Decimal fixed point is different from binary floating point in more ways than base.

Re:We don't need data scientists, we need data engineers by hntrader on Thursday January 14, 2021 @03:58PM (#25782403)
Quants don't care about floating point precision in research. It's just applied stats
Re:We don't need data scientists, we need data engineers by proverbialbunny on Thursday January 14, 2021 @07:36PM (#25785287)
I do, because the results from my research varies when I'm validating the model.
Re:We don't need data scientists, we need data engineers by greenshackle2 on Thursday January 14, 2021 @04:09PM (#25782566)
> If they're using a dynamically typed language to do monetary calculations, it's not going to be ideal.

And yet, Q.

Re:We don't need data scientists, we need data engineers by singhrac on Thursday January 14, 2021 @02:49PM (#25781468)
> If they're using a dynamically typed language to do monetary calculations, it's not going to be ideal.

I think this is an inaccurate take. No one in finance is doing accounting or model estimation using Python's floats; they are using numpy's float32 (or float64) type instead. I think a more accurate version of what you're saying is that static type checking is useful when modeling complicated contracts; this might be true, but I think it's not that important, as those things aren't that liquid anyway.

Jane Street's decision to use OCaml is almost as much about hiring and history as it is about language features.

Re:We don't need data scientists, we need data engineers by twic on Thursday January 14, 2021 @02:53PM (#25781543)
> No one in finance is doing accounting or model estimation using Python's floats

We are. When your input data only has five significant figures, and probably less than that of real information, numerical accuracy is the least of your worries.

Re:We don't need data scientists, we need data engineers by aldanor on Thursday January 14, 2021 @03:26PM (#25782007)
Or, they're using ints instead, at least for market data.
Re:We don't need data scientists, we need data engineers by proverbialbunny on Thursday January 14, 2021 @03:28PM (#25782028)
Fixed precision types technically. Internally they are an int under the hood, so yah basically that.
Re:We don't need data scientists, we need data engineers by chadash on Thursday January 14, 2021 @02:16PM (#25780855)
> "To solve this problem many firms have their own internal programming language."

Any examples other than Jane Street?

Re:We don't need data scientists, we need data engineers by anonymousDan on Thursday January 14, 2021 @02:50PM (#25781493)
Goldman Sachs (Slang)