Re:Engineering is the bottleneck in deep learning research
by PaulHoule on Wednesday January 18, 2017 @10:58AM
(#13427250)
I have seen some painful C and Python written by luminaries. On the other hand, the MiniSAT source code is 500 lines of beautiful C++.
Re:Engineering is the bottleneck in deep learning research
by leecarraher on Wednesday January 18, 2017 @10:27AM
(#13426959)
wrong thread, take the best programming language fight elsewhere
Re:Engineering is the bottleneck in deep learning research
by AIMunchkin on Wednesday January 18, 2017 @10:39AM
(#13427056)
It's not a best programming language fight. In my experiences in the industry, an enormous of amount of technical debt and operational inefficiency is accrued when someone ignorant of how machines and processors actually work (SIMD, cache, pipelines, threading, etc) is in a leadership position to dictate the toolset for solving problems.
This wasn't a noticeable issue until about a decade ago. But it is now and it continues to get worse IMO. The "programming language" bit is just one of its symptoms when the root cause is ignorance of practical computer architecture.
That said, the mentality of throwing all big data problems at Hadoop clusters with 4 year-old GPUs and flaky 10 gB interconnect (many of which could be solved faster on one.big.modern.and.cheaper.machine(tm)) is working wonders for my Amazon stock so maybe I should just shut up and get rich?
I have sat through too many presentations obsessing on HW-level perf/W especially w/r to Deep Learning ASIC wannabes. Just writing one's code in C/C++ (and doing it well) guarantees at least a 2x improvement over Java and a 10-100x improvement over Python. I won't even bring up the computational coup that is CUDA.
But hey, let's base a mobile phone OS on Java and block low-level access to its GPU, that's a fantastic idea, right?
See also many experiences with data scientist and CS primadonnas dismissing low-level coding as "ops." I liken this to the Eloi dismissing the Morlocks as "the help."