Myths of Enterprise Python

Re:Myths of Enterprise Python by frownie on Tuesday March 24, 2015 @09:24AM (#9256360)

I have a fairly large code base (80K LOC). When the size grows, lack of typing can become a problem. 95% percent of the time, the code is explicit enough. However, when you end up with meta code, then it can become really difficult to track the types down in the n-th level of recursion...

Python2 to 3 migration is not easy. There are tools but the problem is that they don't see everything and therefore, you end up with 95% of your code converted. Then it's up to you to figure out the last 5%, which is an order of magnitude harder that the first 95%... So I ended up having a fairly long transition of migrating Python2 code to Python2+3 code.

For bothe these issues, the common discourse is : have proper test coverage. But well, we live in a real world, and maintaining a code coverage strong enough to allevaite the problems (around 90%) is just very hard. If you're coding alone, that may be just too much (my case). In a team setting, with money to spend, that may be possible, but you'd need a very disciplined team.

But anyway, AFAIC, working with Python is just super productive (I compare to Java). It also feels much more battle tested than, say, Ruby.

For me python is not a scripting language but a "glue" language set in the middle of a huge libraries ecosystem.

Now, I didn't do XA transaction stuff, high performance stuff, etc. For me it's more alike a well done Visual Basic : you can achieve a lot very quickly. Contrary to VB, the language and its ecosystem are really clean.

I'm lovin' it

[ Parent | on HN ]

Re:Myths of Enterprise Python by jeffasinger on Tuesday March 24, 2015 @09:49AM (#9256492)

I work on a team of three that has a project slightly larger than you 100k-110K LOC (and growing by about 5k LOC a week), we've managed to keep test coverage at about 95%, and have found it's worth the investment upfront, as it makes refactoring so much easier.

Looking back, I don't think even for a personal project, I would ever do something that wasn't a one-off without good test coverage. It's essentially taking on technical debt, as it makes you much more afraid of fixing anything.

[ Parent | on HN ]

Re:Myths of Enterprise Python by hogu on Tuesday March 24, 2015 @11:23AM (#9257166)

I developed this library ( https://github.com/hhuuggoo/thedoctor ) to help deal with this problem. My belief is that type validation is only part of the problem - being guaranteed about properties of your inputs and outputs is also necessary (for example, make sure the input dataframes have datetime indices, and include the following columns, or ensure that this matrix is nonsingular). I have worked on some of the largest python projects at nyc investment banks, I think I've seen what I can safely call the scariest python project in the world

There is also another library out there which approaches the problem a bit differently

https://andreacensi.github.io/contracts/

[ Parent | on HN ]

Re:Myths of Enterprise Python by crdoconnor on Tuesday March 24, 2015 @10:58AM (#9256972)

>I have a fairly large code base (80K LOC). When the size grows, lack of typing can become a problem.

I do as well, and I find that while we occasionally run into typing bugs, the tests nearly always catch it and they catch it quickly. Moreover, these are tests that we'd write in any language, and static typing would, in most other languages, mean more verbose code. Overall we still come out ahead.

>However, when you end up with meta code, then it can become really difficult to track the types down in the n-th level of recursion...

Yea, I try to avoid that. If there's a library that does meta-code that's unit tested to hell and back, maybe. If I have the time to write one and unit test it, maybe. But, I still try and keep regular projects clear of it.

[ Parent | on HN ]