I re-implemented the techniques described here for a former employer (which sadly means I also cannot release the code). I was able to reproduce the quoted performance figures (both accuracy according to the paper's metrics and execution time), but the actual quality of the output was not very good.

Something like > 90% (number hazy based on vague recollection) of images produced results that had gross, glaring errors that would be unacceptable to a human. Errors like, "We added a horizon line where there wasn't one in the actual image," etc. But a few images produced compellingly plausible results.

It's still very interesting stuff, and some of his later work on using this as an additional prior for more normal structure-from-motion pipelines looked pretty promising. In the form reported in this paper it wasn't something ready to be productized. Although it's also completely possible that I just screwed something up.

The YouTube video in the HN comments links to one of the authors' Stanford page, which redirects to this page at Cornell: http://www.cs.cornell.edu/~asaxena/rccar/

There you can find the source for both the vision code in MatLAB, and a link to http://make3d.cs.cornell.edu/code.html for the full source.

Thanks!

Yeah! Having spent much of the last year plowing through computer vision papers, it's pretty rare to encounter published code or even datasets.

Why is this? Do people spend all their time polishing prose and neglecting the code? Or maybe they keep it closed for commercial intentions? Or is it just a cultural thing in all of academia?

Speaking from experience, the code is normally functional but unsightly, thus it would be embarrassing to place it online. Most researchers are happy to email you the code if you ask for it though.

Most of my code is for scientific computing and consists of a bunch of one-off Jupyter notebooks and Julia scripts that are written with two objectives in mind: get-the-task-done-before-my-advisor-meeting and make-it-as-fast-as-possible. Good coding practices and code cleanliness are not priorities. And particularly for code that needs to run on HPC clusters or supercomputers, it's really hard to make it look nice — one could easily spend weeks designing a proper, modular system only to throw it all away in favor of using a better algorithm that was just published. Anecdotally, the vast majority of my code has produced results that were later discarded.

For example, I have a function named YPXt. What does such a woefully named subroutine do? It multiplies a matrix Y by a permutation matrix P, and then multiplies that by another (transposed) matrix X. The matrices are small, so the overhead of using BLAS for the task is wasteful. It's better to use Julia's macro system to generate the necessary for loops on the fly and let the compiler SIMD-ify the result. Also, permutation matrix multiplies can be done in a way that doesn't actually involve creating a full permutation matrix. And of course the subroutine has side effects — in order to conserve memory I need to overwrite the input matrices instead of allocating new ones. Immutable data structures may work great for functional programming, but the overhead of red-black trees and stateless programming is too high for numerical data crunching.

Add that to the fact that a lot of scientists in academia are unfamiliar with tools like Git, and so manually versioning the code and emailing it back and forth is unfortunately a necessary part of the collaboration process.

I completely agree with you and I can relate with your "objectives in mind". But you really need to push to get others using Git.

GitHub's GUI client is very easy to use and with a bit of nagging around you can get even people who don't understand what VCS are to use it. The one time they ruin the code and get the chance to easily go back to how things were before they'll see its value and endlessly thank you for it.

In my experience, blind reviewers for peer reviewed journal articles in certain cases may argue that publicly available binary or sources are a prior publication of the technique or method, and a reason for rejecting the paper.

This risk is hard to assess in advance. Many journals do not clarify the policy regarding advance publishing of source code or even binaries alone.

Given the impact this could have on a student's career, this is worth considering carefully, and if possible, clarifying in advance with the journal's editor or program chair.

Science = Nature.

Code = Applying Nature.

This would be a lot easier to jibe with if the paper was based on proof alone. A lot of the work in this realm isn't that. Plenty of these researchers are also fine engineers and I'm sure have great code. It just seems crazy to develop a new technique so completely and stop short of releasing the code.

I think it's because the academic culture encourages this as if the code detracts from the purity of the research.

I see where you're coming from. I think the truth is actually dependent on the particular institution and particular academic.

For example, OpenRave is the result of academics open sourcing their research.

Update: Someone posted a link to the code.

I love open source, but there is a very good reason why academics generally don't release their code.

They are scientists: they are best at testing, explaining, and documenting nature.

Code is an engineer's job: making it as fast and robust as possible for the given application or with the latest tools.

They want to make you work for it :)

With all due respect, that is a stupid reason.

I don't have time to implement every last thing. And I don't require robust code all the time -- it's very understandable for the code to not be robust if it's being provided by a small group with not a lot of funding or resources. Hell, if they make it open source I will contribute to it expecting nothing in return to make it better in any way I can.

I really think that the reason the code isn't shared is journals don't encourage this type of thing (or make it easy). Lab websites live and die, so hosting it there is not a good idea. Very simply, the code should should be packed with the paper somehow. I hope some journal takes the lead soon in creating a system that encourages a bundled presentation of paper + code.

Academics aren't always concerned with making your job easy, they have lives too. They may have struggled a lot for their results and their code is still a living document, or they sell it to a company. Maybe their grant ran out and they got pulled into something else. Maybe they died.

There's a lot of factors.

Look at OpenRave. It's the result of an academic open sourcing his research.

I just think at the end of the day, a research paper exists to validate human understanding of nature - not give you a boilerplate for implementing it.