News for hackers. Stuff that matters.
Re:How Lisp-family languages facilitate building bioinformat... by torrent-of-ions on Monday January 09, 2017 @07:06AM (#13355731)
When I started working in bioinformatics I was excited to use Lisp for my work. I wrote a considerable amount of library code and programs at first. But I always found resistance when it came to other people using my programs and even more if the program was to be published and released for general use.

There is a large amount of really, really bad Perl code still in use in biology. Trust me when I say to those outside of the field, I've seen things you people wouldn't believe. I try to discourage Perl use between it makes it so easy for biologists to write terrible code.

I've found a compromise in Python. People will happily use it because it's trendy and ubiquitous. And I think it makes it easier to write OK code. But I miss using Lisp. The style of programming when you have a REPL is perfect for this kind of thing. The ability to quickly write prototypes which can later be improved is an advantage which cannot be stressed enough. C is fantastic when you have a good idea of the algorithms and data structures you want to implement and are concerned with implementing them efficiently. But that's rarely the case when writing bioinformatics code. I need to know if the damn thing works first before I spend time making it work fast. And why would you not use a language where both things are equally easy?

I'm happy to see this paper because it means that I have a way to justify my usage of Lisp in the future. Big thanks to the authors.

Re:How Lisp-family languages facilitate building bioinformat... by yvdriess on Monday January 09, 2017 @07:57AM (#13355996)
A benefit of introducing LISP in bioinformatics is the cultural element that tools are expected to work on the command line using simple linux streams for I/O. (As well as a systemic allergy to dependencies, preferring to do everything from scratch.) You can make drop-in Lisp replacements of shitty Perl tools and no one will be the wiser, as long as you are producing bit-compatible results. I imagine establishing trust in your tool will be harder if you provide them with Lisp vs. Perl source.

Other communities (e.g. statistics, machine learning) have a culture of ramming all their tools in a toolbox-library/framework (R, sklearn). This makes for a much tightly coupled environment, making it much harder to introduce Lisp work.

Re:How Lisp-family languages facilitate building bioinformat... by open_bear on Monday January 09, 2017 @07:29AM (#13355832)
I think that IPython is one of the best tools for scientists. Jupyter Notebooks that allow reproducible results, a lot of data-wrangling libraries, visualizations, distributed computing. I wish more scientists knew of IPython.
Re:How Lisp-family languages facilitate building bioinformat... by svwobwbou on Monday January 09, 2017 @09:03AM (#13356328)
yes - and the thing that makes python approachable is the syntax and standard formatting. I love LISP, I'm not confused or annoyed by the parens, but many people are. LISP would be more popular if there was a good (browser-based) hybrid structural editor with the keyboard-navigation familiarity of a text editor, in a notebook format where the LISP code is displayed like Python (the parens hidden, and handled automatically, but serialized as normal LISP code behind the scenes). This is not that hard to do, and we're nearly there:

* http://gorilla-repl.org/

* https://github.com/kovasb/session

(Also my own early abandoned experiments: http://celeriac.net/iiiiioiooooo/ http://celeriac.net/iiiiioiooooo/public/ http://celeriac.net/iiiiioiooooo-dom/ http://celeriac.net/io/public/)

Re:How Lisp-family languages facilitate building bioinformat... by open_bear on Tuesday January 10, 2017 @01:10AM (#13363005)
GorillaREPL is very nice. There is also a Proto-REPL that looks very promising (https://github.com/jasongilman/proto-repl)
Re:How Lisp-family languages facilitate building bioinformat... by greggyb on Monday January 09, 2017 @02:49PM (#13359202)
See Wisp Scheme: http://srfi.schemers.org/srfi-119/srfi-119.html

It doesn't address the browser-based point.

Re:How Lisp-family languages facilitate building bioinformat... by xaa on Monday January 09, 2017 @04:38PM (#13360040)
Also in bioinformatics, and I use Python too, but not for the same reasons. Among languages with mature statistics/ML/numeric libraries, your choices are R and Python (MATLAB/Mathematica if you're willing to consider non-open-source). R is just terrible, so Python is all that's left if you want a sane language with batteries.

I used to write a decent amount of Clojure. I would be surprised if you would have problems with reviewers complaining about a nonstandard language -- unless you're publishing a library, then you might rightfully get dinged because the target audience would be small. But if you're just running a routine analysis, I have always found the reviewers don't care what language it's in.

Anyway, I dropped Clojure because the numeric support is just terrible. There's Java libraries, of course, but you lose the conciseness and functional aspect. You just can't performantly use arrays or matrices as if they were generic sequences, as appealing as it appears from a distance.

Considered Racket, but I consider lazy evaluation a must (a la Python's generators), and I could never figure out how to use Racket's lazy mode.

So, Python it is... Maybe choosing a programming language is like marriage: better to have something you can tolerate for a long period with all its warts than something you're infatuated with.

> I have a way to justify my usage of Lisp in the future

Haha, OK. I found this paper really unconvincing. No mention of the terms "numeric" or "array", and the only mentions of "statistics" and "machine learning" are in the form of "Lisp would be a really great foundation for stats/ML if someone would just write a library for it".

Re:How Lisp-family languages facilitate building bioinformat... by abrax3141 on Monday January 09, 2017 @06:13PM (#13360787)
Why would one need to mention numeric arrays? Lisp has all the usual capabilities in this area, and with a little effort is as fast as C, sometime faster. See stats comments elsewhere in this thread.
Re:How Lisp-family languages facilitate building bioinformat... by hatmatrix on Monday January 09, 2017 @07:34AM (#13355867)
Building a community around lisp is tricky. While there are many criticisms to this article [1], there are some elements of the "Lisp Curse" which is true - each programmer tends to rewrite his/her own libraries and so a collaborative framework seldom emerges even while many programmers may work in the same domain. This can be a disadvantage in a domain like the natural sciences where deep collaborations are necessary.

Furthermore, the disconnect with traditional mathematical syntax has also turned off many users. There was a predecessor to R called Lisp-Stat [2] which never achieved the acceptance that S/R did, and the Lisp-based syntax of Lisp-Stat was cited as one of the reasons.

Following the example of a high-profile Lisp supporter like Peter Norvig who accepted Python as a "Lispy" alternative early on, I think there are a lot of us who have come to accept Python as a necessary compromise. While Python is not homoiconic, functional programming is encouraged through its libraries rather than its core language definition, and the power of macros are missing, the inherent popularity and all the benefits that come with it is tough to beat.

[1] http://winestockwebdesign.com/Essays/Lisp_Curse.html

[2] http://homepage.divms.uiowa.edu/~luke/xls/xlsinfo/xlsinfo.ht...

Re:How Lisp-family languages facilitate building bioinformat... by throwanem on Monday January 09, 2017 @07:58AM (#13356006)
> each programmer tends to rewrite his/her own libraries and so a collaborative framework seldom emerges even while many programmers may work in the same domain

So, exactly like bioinformatics, then. Trivial tasks are supported by packages of libraries like BioPerl, but all the original research is sui generis, with the attitude in general being that each problem is unique and only results get published, so who cares if the code is a write-only mess as long as it does the one job after which no one will care about it anyway. In such an environment, the "Lisp Curse" already exists anyway, so why not get the benefit of a more powerful and expressive language when you're already paying its cost?

(Granted, Lisp for bioinformatics is never going to become a generally accepted thing. But if you're not releasing code anyway, no one cares what language you write it in...)

Re:How Lisp-family languages facilitate building bioinformat... by GFK_of_xmaspast on Monday January 09, 2017 @11:42AM (#13357601)
> who cares if the code is a write-only mess as long as it does the one job after which no one will care about it anyway

This is exactly the kind of attitude that projects like 'Software Carpentry' (https://software-carpentry.org) are pushing back against.

Re:How Lisp-family languages facilitate building bioinformat... by hatmatrix on Monday January 09, 2017 @08:46AM (#13356244)
What about Bioconductor and Openbabel/Pybel?
Re:How Lisp-family languages facilitate building bioinformat... by throwanem on Monday January 09, 2017 @09:01AM (#13356319)
R and especially Python didn't see a lot of use where I was; the former was regarded mainly as a chart generator, while the latter had one diehard fan (an engineer like me, not an investigator) and no further uptake. It has been a few years since I was there, though, and I haven't kept closely in touch, so perhaps things have changed in the interim.
Re:How Lisp-family languages facilitate building bioinformat... by torrent-of-ions on Monday January 09, 2017 @09:17AM (#13356411)
> and the Lisp-based syntax of Lisp-Stat was cited as one of the reasons.

The syntax referred to is S-expressions. They are a simple and elegant way of writing trees. Lisp was originally supposed to have M-expressions which would be more like mathematical notation, but it was found that Lips users actually liked S-expressions.

The reason becomes obvious after some amount of Lisp exposure. S-expressions are so simple that before long one does not even see them any more. The parentheses give way and one can just look straight at the Lisp code itself. For this reason many Lisp programmers do not even edit the S-expressions. They instead use something like paredit to edit the underlying tree directly.

It's similar to how readers don't really "see" the letters that make up a word in many natural languages. And, further, they don't see the words that make up certain expressions. We have this fantastic ability to internalise language but so many are choosing not to use it when it comes to programming.

Re:How Lisp-family languages facilitate building bioinformat... by cicero on Monday January 09, 2017 @09:42AM (#13356604)
Until children are taught S-expressions in elementary school math, infix math notation will always be more familiar to the general population, and there will always be a learning barrier that must be overcome to gain similar skill with S-expressions, no matter how superior they might be.
Re:How Lisp-family languages facilitate building bioinformat... by JaumeGreen on Monday January 09, 2017 @10:40AM (#13357070)
But we are all using S-expressions without noticing. What do you say "Add 3 and 4" or "Take 3 and add 4"?

That is the same with most things, we say the verb before the object(s).

Re:How Lisp-family languages facilitate building bioinformat... by cicero on Monday January 09, 2017 @01:41PM (#13358605)
Your English notation is not really an S-expression. I would say it is similar in that the operation comes first. Even though people understand such notation for simple expressions, most people are not used to doing advanced (beyond elementary school) math using such notation.

I agree that the concepts behind S-expressions are simple and logical, but expressing an entire program using S-expressions is an additional thing that must be learned. Other languages chose a syntax that is closer to what people are used to and that helps with the adoption of the language, even if the language loses other capabilities.

Re:How Lisp-family languages facilitate building bioinformat... by abrax3141 on Monday January 09, 2017 @01:38PM (#13358581)
Yes, that right. And moreover, much math isn't infix: Factorial is postfix. Sum, Product, and Integral are prefix. Infix is actually the exception, and is not semantically uniquely without parens anyway (languages have different orders of operation!), so folks should just get over the mindless prefix hate.
Re:How Lisp-family languages facilitate building bioinformat... by taejo on Monday January 09, 2017 @11:31AM (#13357525)
"What is 6+1*0-2/2?" memes/threads on Facebook have taught me that infix math notation is more or less completely unfamiliar to the general population. At least sexprs are fully parenthesized.
Re:How Lisp-family languages facilitate building bioinformat... by cicero on Monday January 09, 2017 @01:49PM (#13358690)
If you are talking about order of operations, all of the sixth graders that I teach know how to evaluate that expression. While it's true that I had to teach them * means multiply and / means divide, they can quickly pick that up. Our school is all honors and the students are above average intelligence, but they are not all math or tech geeks.

On the other hand, I have taught courses at the high school level at the same school based on "How to Design Programs" [1] using the Racket language. Although I can explain the concept of (+ 6 (* 1 0) (/ -2 2)) rather easily, the students make mistakes writing expressions for several weeks.

1: http://www.ccs.neu.edu/home/matthias/HtDP2e/

Re:How Lisp-family languages facilitate building bioinformat... by taejo on Tuesday January 10, 2017 @04:19AM (#13363647)
I wasn't entirely serious, and I believe sixth-graders can evaluate infix expressions with correct order of operations, but it was adults I was referring to, many of whom have long forgotten.
Re:How Lisp-family languages facilitate building bioinformat... by agumonkey on Monday January 09, 2017 @07:21PM (#13361343)
I can't take the infix arithmetic argument anymore. In any interesting piece of software, such arithmetic will only be 5% of the problem. You'll deal with many other concepts, combinators, morphisms, trees, vector spaces, geometry ... which are often functional in syntax (Ker <morphism>) ...

Plus when you're used to trees, linearization and parsing, you *fix the way you see fit, pre post or in ...

Re:How Lisp-family languages facilitate building bioinformat... by DonaldFisk on Monday January 09, 2017 @02:02PM (#13358804)
Lisp-Stat (or XLispStat) didn't just use Lisp syntax, it was a Lisp.

I've used XLispStat, S-PLUS (a commercial version or R), and SAS. SAS had the most comprehensive statistical libraries, but I never liked the language. I liked xlispstat but the statistical libraries weren't as good as those of SAS or S-PLUS. For data mining, I settled for S-PLUS, which was sufficiently lispy, and had a REPL. So it wasn't just the syntax (or lack of it).

Beyond being dynamically typed and having a REPL, Python isn't particularly Lispy: it's less functional, and it has no (linked) lists or symbols. It's quicker to develop in than most of the non-Lisp alternatives, though.

Re:How Lisp-family languages facilitate building bioinformat... by nerdponx on Monday January 09, 2017 @10:07AM (#13356804)
You might be interested in Hy [1], which is a Lisp implemented in Python that can import Python libraries. I'm not sure if it cross-compiles to Python or just to Python bytecode, though.

1: https://github.com/hylang/hy

Re:How Lisp-family languages facilitate building bioinformat... by willismichael on Monday January 09, 2017 @11:58AM (#13357737)
For a moment you had me wondering if there would be a way to use Hy and google's new Grumpy together, and then I dismissed the idea as probably utter madness. For a long time I've been pining for a Clojure-like lisp that can compile to machine code.
Re:How Lisp-family languages facilitate building bioinformat... by nerdponx on Monday January 09, 2017 @01:32PM (#13358521)
Why wouldn't it work?
Re:How Lisp-family languages facilitate building bioinformat... by willismichael on Monday January 09, 2017 @03:19PM (#13359446)
I'm not certain that it wouldn't work, but it would have multiple compile steps: Hy => Python => Go => Native, which would either cause a lot of pain or make nigh impossible many of the advantages of using a lisp, like connecting a repl to a running process and being able to evaluate and rewrite code from there.
Re:How Lisp-family languages facilitate building bioinformat... by nerdponx on Monday January 09, 2017 @04:52PM (#13360152)
You could prototype in the Hy REPL then compile after.
Re:How Lisp-family languages facilitate building bioinformat... by agumonkey on Monday January 09, 2017 @10:12AM (#13356853)
IIRC it's sexp to python AST.

There's also Pixie, a python lisp to LLVM system, that was specifically made for high performance.

Re:How Lisp-family languages facilitate building bioinformat... by nerdponx on Monday January 09, 2017 @11:53AM (#13357697)
Re:How Lisp-family languages facilitate building bioinformat... by agumonkey on Monday January 09, 2017 @01:02PM (#13358243)
It seems so.. which makes my LLVM mention false. There's a talk by the author online http://www.youtube.com/watch?v=1AjhFZVfB9c

worth it

Re:How Lisp-family languages facilitate building bioinformat... by GFK_of_xmaspast on Monday January 09, 2017 @09:29AM (#13356502)
> I've found a compromise in Python. .... But I miss using Lisp. The style of programming when you have a REPL

95% of my python programming time these days is spent inside a repl.

Re:How Lisp-family languages facilitate building bioinformat... by gwu78 on Monday January 09, 2017 @11:48AM (#13357642)
What Lisp did you use?