PowerShell does pretty much all the things you mentioned except the lispy part. Actually, I guess it has first class and anonymous function support so if you squint a little you have the lispy part as well.

PowerShell is a great source of inspiration (I went through a PowerShell reference book before I started Elvish), but IMO it's too complex and has too many (avoidable) pitfalls.

I don't know much about PowerShell, although I've heard it's good but somewhat verbose (something I can't stand). I should probably read more about it to see what lessons it has, but of course, as an OS X user, I can't use it.

Xerox Parc Interlisp-D, Smalltalk and Lisp Machines cover all of that and then some.

Their REPLs are more powerful than UNIX shell, you get the whole OS at your disposal using the same language.

The REPLs are graphical, similar to the ideas being brought back by IPython.

Combining objects or functions on the REPLs is Zen compared to the simple UNIX pipes.

> - Nonzero-exit modeled as exceptions. (I think elvish is doing this, but I haven't reviewed the code in detail.)

No really. I have not yet decided how error/exception (http://www.haskell.org/haskellwiki/Error_vs._Exception) handling should work.

I actually like how golang does it - errors with panic/recover, exceptions with return values. But I have not been able to make a decision yet - the situation is a bit more complex than golang. As a result there is no if/else builtin yet, let alone raise and try/except (or panic and recover).

For the record, I am personally usually a fan of making exceptional conditions return values (though I prefer Rust's explicit unwrapping to Go's approach), since in most programs it's good to think at least somewhat carefully about error handling anyway. However, in one-liners, there is no need for proper error handling, and the vast majority of exceptional conditions (e.g. file not found) are caused by mistakes that should be rectified by changing the code. Therefore, in a language that prioritizes one-liners, as I believe any shell should, I believe the traditional exception approach is the best.

> - A story for passing typed data around between processes (potentially in different languages) rather than just builtins. I'm not sure exactly what the story should be, but it should exist.

I haven't thought much about FFI yet. But since in shells you interact with other programs so much, adding FFI would a worthy attempt.

> - A somewhat more succinct syntax, with metaprogramming kept in mind.

Coming up with syntax is actually the hardest thing I find when writing Elvish, since there is much more syntax constraints for a shell. Would be happy to hear advices.

> - A JIT, eventually.

Maybe later.

Remember this?

http://en.wikipedia.org/wiki/Common_Object_Request_Broker_Ar...

By these guys?

http://www.corba.org/

Whatever happened to that?

It's been replaced by dbus. Which does indeed have strict types and other features that might be desirable (multicast?). You raise an interesting idea: A shell that pipes materials around from process to process through dbus, and a set of command line tools that accept input over dbus.

http://en.wikipedia.org/wiki/D-Bus does not work for IPC between computers, AFAIK, only for processes within one machine. CORBA works across machines.

It was terrible on multiple levels and deservedly faded away after a short fad.

And this:

http://google.com/search?q=xml-rpc

Still being used some.

I think the points about types are really interesting. It sounds like a worthy project for that alone.

Does there exist anything like 'type protocols' across languages? Would it be feasible?

Well, there's JSON and variants. Probably a lot simpler than what you had in mind, but if Unix pipes were based on JSON (and had a suitable shell language to work with it), that would already make many tasks easier, like

- Grep the output of ps, without having to count column numbers or do special work to preserve the header line after filtering.

- Search for files matching some conditions, without remembering the weird and unique syntax of find(1) - instead you would write something vaguely along the lines of

    ls | filt {< $1.size 500} {eq $1.type file}

where the stuff after the pipe is the shell language, so theoretically more memorable than

    find -size -500c -type f -maxdepth 1

As a bonus, if you want to cache the output for multiple searches rather than going through the the kernel every time (OS X find is rather slow on huge source trees), you can save the output of ls to a file and keep the rest the same. Good luck doing that with find.

- Handle filenames with spaces and newlines without issues. Simple, but newlines at least are near impossible to solve in the standard shell environment - only a few select tools support null-separated lists. Spaces are easier... unless you want such extravagances as multiple filename columns in a table.

Of course it's not that simple: structured data can sometimes be hard to work with and formatting for display is an issue; Unix's "everything should be plain text" philosophy didn't come out of nowhere.

But I honestly believe that it's time to reexamine that tenet.

Let's take a concrete example: Say you have a programming language L that has a parser that is implemented in lang. A, and an interpreter implemented in lang. B. Now if you want the interpreter (in B) to interpret the program, you need some structured data to come from the parser (in A). Abstract syntax trees (ASTs) are the usual conceptual tool for this. Now assume that the parser outputs an AST that is described by something, like for example an algebraic data type (ADT). Assume that the structure of the ADT conforms to some type protocol. Now the interpreter can implement the ADT in its language (B) and thereby take the ADT directly as input. Now you can use a "typed pipeline" to easily combine these two programs:

> parser directory/file.lang | interpreter

Does this make sense? Would it have some utility?

RPC has to solve the type protocols issue often. Eg Google's protocol buffers.

Protobuf.

Why do we need a new 'real programming language'?

Because it's both a real programming language and a shell, which is rare.

Yes, this is the main reason for me. Where being a shell has two parts:

First, it's universal. Python and Perl and Java all have huge ecosystems, but one of the few places where programs written in all those languages can come together is the shell. By "real programming language" I don't mean some monstrosity with its own packages and whatnot. I just mean a simple principled language to tie things together that doesn't feel like a Turing tarpit.

Second, its highest priority is efficient interactive (REPL) use (although, as a "real language", it should be much better than bash for scripts as well). Some languages have good REPLs, but most don't, and there are important design considerations for them.

For example, if you're banging out a one-liner where you have some data and make a series of transformations on it - thinking through each step in turn - you don't want to have to write functions in reverse order of application, third(second(first(data))). With pipelines, it's forward, first | second | third. In a batch program (in both imperative and functional languages, though more often in imperative), you might avoid this by performing each step as a separate statement, assigning each result to a descriptive variable. But interactively, who has time for variable names? (The elvish author makes a similar argument in the readme.) Yet few languages support this outside of OOP dot syntax, which only works in a subset of cases.