News for hackers. Stuff that matters.
Re:Giving up on Julia by SolarNet on Friday May 13, 2016 @06:30PM (#11693615)
> - Slowing down in development

I think part of this problem is the community, multiple people have reported having bad interactions with core language devs. Also the policies for inclusion of features, how to propose features, how decisions are made with respect to the code base, etc. all seem poorly documented.

Re:Giving up on Julia by KenoFischer on Friday May 13, 2016 @07:56PM (#11693932)
> I think part of this problem is the community, multiple people have reported having bad interactions with core language devs.

I can only think of a couple of instance where I've heard anything like this.

The first and most prominent is Dan Luu. I'm very sad about losing him from the community. I've met him a couple of times in person, I still read his blog and I very much respect him. I don't think any of us have the full story of what happened there, but I sincerely hope that time may be able to smooth things over.

The second was in an HN comment that I just went back to find, and discovered that is was you. I'm sorry if you had a bad experience with the community. I know my perspective is biased, but I've rarely interacted with a community that's as passionate, helpful and friendly as the julia community (LLVM is up there as well).

As for documenting the process, I agree there could be some improvement there. We do have a contributors guide (CONTRIBUTING.md in the main repository). As for policies for feature inclusion, we've considered having more formal code owners for parts of the code base (right now there's mostly de facto code owners for various parts), but haven't found it necessary so far. I definitely expect some of this to be discussed at JuliaCon in June.

Re:Giving up on Julia by SolarNet on Friday May 13, 2016 @08:30PM (#11694026)
I remember plenty of cases I got simply from observing the Julia repo for a few months, I could list them if you like... But I've been told by Julia contributors before that discussing this on HN is not an appropriate place. So we can discuss it, where I start linking to public examples on HN, or you can ignore my opinion on a problematic community and we can not discuss it.

As to the code standards, my problem isn't with the CONTRIBUTING.md; that seems fine. It's how people make long term suggestions about fixing lasting problems. As an example people who suggest refactoring large sections of Julia's expansive base library (1600 symbols? a problem that causes parts of the slow startup time and memory usage) into other libraries (which would improve batteries included, etc. features) are summarily ignored / shouted down / (predicated on my previous paragraph) banned. Important improvements to the language go in circles (often on JULEP tagged issues) especially any large improvement that would require multiple people and lots of dev work and seem to gain little traction unless a core dev just takes the time to just do it; rather than a timeline to reach consensus and then design a plan for implementing it.

As smaller examples. There was a push to increase test coverage and then months later commits plummeted the code coverage numbers and no one seemed to care. Sections of important code are uncommented, undocumented, and (mostly) untested on purpose. There are modularity problems and software architecture limitations inherent in the language design that go unaddressed. Unpredictable un-typed exceptions, namespaces/modules being hard to use for non-trivial designs, "interfaces" that aren't really. And these are just the ones without serious work behind them I can remember off the top of my head (the debugger and threading I acknowledge being worked on seriously). Though things being worked on seriously suffer problems as I mentioned earlier.

Issues like these cause many of the original poster's problems. Julia is a great language, but the community seems unable to take the steps it needs to to actually make a "Tier 1" general purpose language (e.g. like python, go, clojure etc.). Setup a serious set of policies regarding language proposals; setup a transparent community policy and actually follow it; acknowledge that certain development idiosyncrasies aren't going to fly anymore; take software architecture concerns seriously; remove the toxic core language dev (that to my count has cost you at least 4 serious contributors if not more).

Julia is likely "slowing down" relative to the expected development (which many people have observed) likely (by my thinking) because it is failing to attract/retain the developers it should be as a promising language (in my opinion) because of it's poor development and community standards.

But hey, these are just my suggestions, I don't have any skin in this game. I just wish Julia was as good as it promised to be, and I'm a bit bitter it isn't for human, rather than technological or financial, reasons.

Edit: Sorry I expanded, it felt unfair to not be more complete about it.

Re:Giving up on Julia by KenoFischer on Friday May 13, 2016 @09:39PM (#11694209)
Also, I do thank you for taking the time to give feedback.
Re:Giving up on Julia by KenoFischer on Friday May 13, 2016 @09:29PM (#11694187)
> I remember plenty of cases I got simply from observing the Julia repo for a few months, I could list them if you like... But I've been told by Julia contributors before that discussing this on HN is not an appropriate place. So we can discuss it, where I start linking to public examples on HN, or you can ignore my opinion and we can not discuss it.

I do think HN is not the right forum to link to individual comments and call out people for their behavior. Perfectly happy to discuss the technical issues though. If you would would still like to discuss, or at least bring such instances to my attention, please do feel free to send me an email.

> People who suggest refactoring large sections of Julia's expansive base library (1600 symbols? a problem that causes parts of the slow startup time and memory usage) into other libraries (which would improve batteries included, etc. features) are summarily ignored / shouted down / (predicated on my previous paragraph) banned.

There are several recent examples where people have suggested such things, e.g. They were neither ignored nor shouted down nor banned. https://github.com/JuliaLang/julia/issues/16357 https://github.com/JuliaLang/julia/pull/16070

> Important improvements to the language go in circles on JULEP tagged issues namely any large improvement that would require multiple people and lots of dev work seems to gain little traction unless a core dev just takes the time to do it; rather than a timeline to reach consensus and then design a plan for implementing it.

I don't understand what exactly the criticism is. At first I thought it was about, too much dicussion, but then it seemed to be about too little dev time, please do clarify.

> There was a push to increase test coverage and then a few months later commits plummeted the code coverage numbers and no one seemed to care.

I'm not really sure which instance you're referring to, but last time there was a major drop in coverage, it was investigated, found to be a bug in the instrumentation (which was fixed). I don't really think it's fair to say that people don't care. There's somewhat of a tooling problem here, since we can't run the coverage tests on travis, so we don't get them integrated in the GitHub UI, but people do look at them and add tests as appropriate.

> Sections of important code are uncommented, undocumented, and untested on purpose.

Yes, there are undocumented and uncommented sections in the code base, but I wouldn't say that they are so on purpose. I do admit to having added such hacks in the past (and not documented them because they were gonna go away soon after), because I needed them in outside packages, but people don't let me do that anymore ;).

> Unpredictable un-typed exceptions, namespaces/modules being hard to use for non-trivial designs, interfaces without any sort of type enhancement.

These are fair points, which I'm sure you've seen the issues about. The problem is the availability of developer time, not some sort of unwillingness to fix problems. Putting together the road map of what goes into each release and prioritizing are very hard, because there's just so much that could be worked on.

> setup a transparent community policy and actually follow it;

This is being discussed (as part of larger discussions around community governance) and as I mentioned will likely be a topic at JuliaCon.

Re:Giving up on Julia by ScottPJones on Friday May 13, 2016 @10:01PM (#11694262)
I haven't disappeared (even though somebody just might have preferred that I had done so), I still love the Julia language (even though I think there are some warts, some flab that can be lost, etc.), am still programming in Julia pretty uch 24/7, have been very active in trying to help people new to Julia (on Gitter, StackOverflow, Quora, julia-users), even teaching it to my kids, promoting it on Twitter, trying to help improve the language as much as I can without being able to submit issues or PRs on GitHub, and contributing what I can of my own work under MIT license on GitHub. I'm also attending JuliaCon 2016 this summer, will just have to see what my reception there will be.
Re:Giving up on Julia by SolarNet on Friday May 13, 2016 @11:02PM (#11694385)
> I do think HN is not the right forum to link to individual comments and call out people for their behavior.

Fair enough, my problem is that the project continually looses what appear to be serious contributors to a toxic developer (admittedly all second hand to me; but there are public examples and some private reports I have; including the person who first showed me Julia years ago). For all that it is an interesting language with plenty of problems I would enjoy fixing and taking the initiative on I simply don't have a desire to want to contribute. Again mostly due to politics. Which just sort of makes me bitter you fail to acknowledge the problems.

Like why should I take lots of time (I'm fine doing hard things like reading undocumented code bases if it's worth it (large sections of the core compiler code due to it's lack of comments or documentation are hard to read)) to help you fix technical problems, when you have social problems that are noticeable to anyone paying close enough attention to the community.

Also, thanks for taking the time to listen to my rants. I'm just disappointed.

Re:Giving up on Julia by KenoFischer on Saturday May 14, 2016 @12:38AM (#11694564)
> Which just sort of makes me bitter you fail to acknowledge the problems.

I'm perfectly happy to acknowledge that there have been conflicts in the community. However, I maintain that the overwhelming majority of interactions in the community are immensely positive. Please do also consider that there may be parts of the story that you are not seeing in the public issues. I do hope you will consider giving the community a second chance. You have quite clearly identified some of the major technical challenges we face and we'd love to hear any ideas you have to address them.

Re:Giving up on Julia by SolarNet on Friday May 13, 2016 @10:43PM (#11694344)
> There are several recent examples where people have suggested such things

That's great but amount to a couple of functions, the larger issues addressing more serious refactorings never get off the ground (Base contains multiple thousands of symbols (not even counting how in a multiple dispatch language one symbol can contain hundreds of definitions) common lisp, by all accounts a sprawling language with multiple dispatch, has 978).

> I don't understand what exactly the criticism is. At first I thought it was about, too much dicussion, but then it seemed to be about too little dev time, please do clarify.

Both. Neither. It's about how there is unending discussion until a core dev just does it. This is not an efficient use of anyone's time. Set the scope of an issue, discuss, decide on a course of action, layout issues. This allows a core dev to get their input in place, but then not have to actually wait for them to have the time to do the whole thing themselves.

> There's somewhat of a tooling problem here, since we can't run the coverage tests on travis, so we don't get them integrated in the GitHub UI, but people do look at them and add tests as appropriate.

The number seems to fluctuate randomly from 83% in January, to 11%, 45%, 65%, 75%, 11%, 81% now. Which is a net loss and wholly sporadic. How does anyone know what to write tests for when coverage bounces all over the place? Also what good are the tests if no one takes the time to use them anyway? You have tests but they seem pretty useless with how you are using them. I understand it's a tooling issue, but a metric isn't very good if no one is bothering to use it, why waste the time to not use it.

> but I wouldn't say that they are so on purpose

"Look at the code and you'll understand." is the jist of the documentation of a 10000+ SLOC base used at the heart of Julia. Sure I could read all that, but I'd rather have a bit of documentation. Also the tests for this amount to it's examples.

> not some sort of unwillingness to fix problems

I disagree on at least 1 of them. Typed exception handling was flat out rejected because it's too much like Java by key devs; forget the fact everyone does it anyway using reflection. Interfaces and modules don't seem to make any progress because of some people complaining about similar issues.

> Putting together the road map of what goes into each release and prioritizing are very hard, because there's just so much that could be worked on.

Then stop wasting dev time and work on the community policies that make it easier to retain devs.

Re:Giving up on Julia by sgt101 on Saturday May 14, 2016 @03:03AM (#11694850)
I can only speak for myself, I attended an evening "getting started with Julia" event at MIT a few years ago, the devs there were very tolerant of me not being able to do some of the exercises and sat with me and helped me understand. Then they gave me Pizza. I've moaned sometimes about some of the decisions that they've made and they have always responded to me and showed me that they have a good reason for doing what they are doing - even if I still think that might be wrong.
Re:Giving up on Julia by chrispeel on Friday May 13, 2016 @10:36PM (#11694329)
I have never had a bad interaction with the core language devs. OTOH, they are passionate about the language; I've found all the core developers I've met very helpful.
Re:Giving up on Julia by SolarNet on Friday May 13, 2016 @11:31PM (#11694449)
I'm referring to one specific core dev. However they all know each other from MIT so it makes it difficult for them to deal with that fact. I can point to 3 public examples, in addition to a couple of private reports about problems which allow me to be certain. But admittedly it's all second hand to me (from people who work close to the language).
Re:Giving up on Julia by davidacoder on Saturday May 14, 2016 @12:35AM (#11694561)
I've been using julia for a little more than two years now. I've been subscribed to all the mailing lists and regularly read the github issues. Not all of them, but a fair share. I've never, ever seen any behavior by any of the core devs that one could even remotely describe as toxic, rude or anything like that. The community is actually really helpful and supportive.

I have seen this point about a toxic dev been made before in some blog post. Back then I followed the story up because it seemed so at odds with my experience with the core devs. It took a bit of googling and following links, but in the end, in my mind, there was simply nothing to the whole story. The supposedly rude behavior was not at all rude, imho.

I don't really know where these allegations come from, but I find this kind of "I've heard second hand that there is a toxic dev" inappropriate. If someone has a problem with someone, make it explicit, post the email that you dislike, so that others can judge themselves. But these vague accusations are not helpful, and at least from my point of view entirely at odds with how I have perceived the behavior of the core devs over the last two years.

I should say that I'm not part of the MIT crowd. I've never met or talked with any of the core devs and don't know them beyond reading their emails on mailing lists and sporadic interactions on github.

Re:Giving up on Julia by ScottPJones on Saturday May 14, 2016 @05:07AM (#11695206)
The rude behavior definitely exists, I can point out a number of examples if you send me your e-mail address (you can look me up, @ScottPJones on GitHub, easily enough). I've seen that that sort of disrespectful behavior tends to spread in the community, unfortunately, when nobody dares call out one person for their comments and actions, because of their position.
Re:Giving up on Julia by chappi42 on Saturday May 14, 2016 @06:20AM (#11695438)
But then - as a bystander - let me mention that I once wondered about your insistent replies in some gh issue. God, can't he stop, I thought, and, wow, they have patience. Fortunately such a situation is rare and I'm very often impressed by the intelligent friendly conversations I see. A lot to learn.
Re:Giving up on Julia by ScottPJones on Saturday May 14, 2016 @09:51AM (#11696114)
Yes, a year ago I'd never worked on any open source project, didn't use GitHub, Google Groups, Twitter, StackOverflow, or any of the other programmer social media sites, and was a total newbie about Julia, using git, and dealing with an open source community. Since I had convinced the other people at the startup where I am working that using Julia would be better for us than a mashup of Python, C, C++14, R or Octave, when I started encountering serious bugs and performance problems in areas that were critical to our development, I did get very insistent, esp. where I had lots of previous experience in the area. Instead of simply complaining about the problems, I submitted issues, wrote documentation, made PRs to fix the issues (in the Scheme parser code, in the core C code, in the C Unicode handling module, esp. to the string handling code in Base, and helped improve the coverage by adding a lot of unit tests). It was a major learning process for me, learning best practices for how to break apart a PR into smaller chunks (even though accepting huge PRs from other developers happens all the time), trying to focus on single issues in a PR, and trying to improve my own communications skills on social media. Between the time I started until early September when v0.4 was released, I'd even reached the point of being the #6 contributor to JuliaLang/julia (#26 all time out of some >400 contributors), so I don't think anybody could possibly accuse me of not having putting my time and effort where my mouth was, as far as fixing problems I encountered. Having gotten the critical issues fixed and into v0.4, so we had a reasonably stable base for our product, I was able to concentrate wholly on our product, and so stopped being so pushy about getting things fixed. ;) I still think there are some real systemic problems that need to be addressed, in how things are done, and hopefully some time will be spent to deal with that at this summer's JuliaCon 2016.
Re:Giving up on Julia by chappi42 on Sunday May 15, 2016 @06:47AM (#11700365)
Thank you for this reply, I also wish you luck with your project, joy at the JuliaCon and I'm glad that 0.4 worked :-)

Maybe a source of discontent is that startups are using Julia 'for real' and are under pressure to deliver. On the other side Julia is still developing. Core devs work feverishly but some issues just need 'time to brew'. Quiet thinking/coding alone or in small circles.

IIRC on a video of Alan Edelman he ~ said that he was involved in HPC for 30 years and to this day they weren't that successful and don't know how to do it. Julia is trying to do it in a completely different way. - Considering 30 years, does it matter if Julia takes 'her' time to be crowned?

Re:Giving up on Julia by ced on Saturday May 14, 2016 @07:49PM (#11698593)
You know, I read some of the harsh comments you got from people, and they may have been right to say these things, I don't know. But kudos for taking it in stride and maintaining a positive attitude. Good luck in your projects.
Re:Giving up on Julia by ScottPJones on Saturday May 14, 2016 @10:03PM (#11699022)
Thanks. I've never had any problems with constructive criticism, no matter how harsh it might seem (often that is where you learn the most). The problem is when people make disrespectful, disparaging, insulting remarks (which is against both the NumFocus & Julia community standards), and nothing is done about it, which has ended up driving away several people already. Hopefully that can be addressed.
Re:Giving up on Julia by davidacoder on Sunday May 15, 2016 @12:38AM (#11699458)
I followed your interactions with the core devs, at least the public ones. I think the core devs handled the situation with an enormous amount of patience and tact.
Re:Giving up on Julia by sgt101 on Saturday May 14, 2016 @03:11AM (#11694873)
Perhaps you might consider adding a note to your first post making it clear that you are identifying a problem with one other person rather than the whole community and that this is all second hand rather than your own experience?
Re:Giving up on Julia by SolarNet on Saturday May 14, 2016 @03:26AM (#11694922)
You realize edit timers are a thing right?