That said, during my research I ran across an ancient Usenet thread from 1989[1]. In it, the OP asks:
> Should the language designers be making work for the language-oriented editor designers or should the language-oriented editor designers be making work for the language designers?
(The thread rapidly devolves into an all out flame war about whether or not C can be considered to be context-free)
Lisp feels like a language designed specifically to make a language-oriented editor designer happy, but most other popular languages fail the context-free test one way or another, thus making them difficult to define good grammars for. The problem seems to be that historically the language designers have far outnumbered the language-oriented editor designers.
[1] https://groups.google.com/d/msg/comp.lang.misc/MCZmQv56--Q/O...
And what do you mean by "s-expressions just too simple"? Isn't simplicity something to strive for?
Funny, that makes me think of the new Wikitext editor that the Wikimedia Foundation is developing. [1]
They are using the previous Visual editor infrastructure, and they are having problems because features that make sense for a rich text editor are creeping into the plain-text code editor (such as unwanted copy-pasting of styling code).
I should stress "a bit". Actually, looking closer at s-expressions right now (I was writing based on what I remembered), I'd like to flip that statement. S-expressions are just a bit more complex than what I have in mind. Or alternately: they're equivalent under some trivial transformation.
It depends on how you look at things - there are supposedly many different implementations of s-expressions, which support different fundamental data types. The basic idea is simpler. There are no fundamental data types, just nodes. For example a 'bit' is a node which can contain one of two child nodes ('one' or 'zero'). Any tree which represents data in memory on a computer can be expanded down to a collection of bits. Though in a text representation or tree editor the user will generally have collapsed the tree such that they don't see individual bits.
What I have in mind could look more complex than s-expression in a different context though: the text file representation of the trees may have more syntactic sugar than s-expressions in lisp.
The representation isn't significantly different, but the focus is. I'm focusing more on things related to type theory, schemas, how to represent patches/diffs, standardizing parsing/generation and other transforms, etc.
I am working on something like that too, and I'm completely fine with symbols (with their arbitrary definition by lisp and user) being the fundamental elements.
I'd say what you gain is reusability, mostly. When you impose a datatype on data, it comes with a series of constraints and expectations, so you can only use the data in the ways prescribed by its type.
If the data doesn't have attached a type of is own, you can use it in different ways at different contexts - this can be valuable for data transformation processes, such as compilation or system interfaces. I suppose you could get the same effect by casting the data to a new type when you change it to a new context.
I've read a bunch about applying semiotics theory to programming, and changing the meaning of the symbols "on the go" is closer to the way we think (inferring meanings from the signs adequate to the current context) than the old mathematical approach of "every datum has one well-defined type, and only one".
PS, if you're as enthusiastic about this idea as I am, we should talk, can I email you?
Ummm json??
JSON is way too complex. JSONS assumes that you have an object/record structure (labels and values), and gives you both objects and arrays with which to build tree structures.
E.g.
What I've concluded, is that we don't have a good representation for a general purpose tree editor to work on. Roughly speaking, S-expressions are just a bit too simple, and XML is way too complicated.
General purpose plain text editors work so well because we've agreed on a common representation (more or less), which is easy for text. But as soon as you want to move to useful, common tree-structures, you have to agree on both representation and semantics, which makes it much harder.
One challenge we need to solve is - what level do you want to work on? Let's say you're working on some code. You may want to treat functions and blocks as a tree structure, but you want to treat simple mathematical expressions as text. Where this threshold is, is entirely context-dependent. The editor needs to understand the language and be able to expand text into its tree structure, or collapse the tree into its text representation, at any node in the tree.
This implies that we need to agree on a common format for defining the conversion (parsing and generating) between text and trees. We'd probably also need a package system which contains common definitions for all major languages.