https://stackoverflow.com/questions/9533139/what-would-cloju...
The point is that macros become much harder once you move away from leading parenthesis.
Apple tried your reasoning with the Dylan language in the 90s - and the Dylan language died.
Programming languages never live or die on their technical merits. Javascript is a hot mess, and yet it is one of the most popular languages in the world.
Also:
> macros become much harder once you move away from leading parenthesis
That is not really true. You do lose the backquote syntax, but that is only useful for writing the simplest most trivial macros. That turns out to be the kind what most people write and so backquote is a big deal, but if you want to write more complex macros (e.g. cl-who) then backquote becomes less useful and the surface syntax becomes less relevant.
This is... an extraordinary claim. Prolog uses foo(hello, world) syntax (note the comma) and has no problem with macros or any other manipulation of such terms. Note that there are longer, better reasoned, and higher voted answers to that StackOverflow question than the accepted answer which opts for "macros would become more difficult" FUD.
I think your best insight here is the importance of the "lid off" intermediate representation; plenty of languages have an eval function of some sort, but it does both READ and EVAL in your terms with no intermediate access to the parsed representation.
(with-db [conn (connect ...)] ...)
To:
(let [db (connect ...)] (try ... (finally (close db)))
Just like:
(defmacro with-db [[sym expr] & body]
`(let [~sym ~expr]
(try ~@body
(finally (close ~sym)))))
The backtick/tilde notation isn’t macro-specific: it’s a generic way to template readable datastructures that I can use anywhere, so when I see it in a macro, it isn’t some strange API to learn, it’s just a handy way to build up the datatypes I use all the time.a 3rd party library that, to be portable, needs to ultimately serialize to text, does not support homoiconicity.
But that's not a requirement for macros. One can write macros from a parsed AST. But that's not what Lisp does.
Languages such as Python and C draw clear distinction between literal values on one hand and flow control statements and operators on the other. Numbers, strings, arrays, structs are first-class data. Commands, conditionals, math operators, etc are not; you cannot instantiate them, you cannot manipulate them.
What homoiconic languages do is get rid of that (artificial) distinction.
Lisp takes one approach, which is to describe commands using an existing data structure (list). This overloading means a Lisp program is context-sensitive: evaluate it one way, and you get a nested data structure; evaluate it another, you get behaviors expressed. The former representation, of course, is what Lisp macros manipulate, transforming one set of commands into another.
Programming in Algol-descended languages, we tend to think algorithmically: a sequence of instructions to be performed, one after the other, in order of appearance. Whereas Lisp-like languages tend to encourage more compositional thinking: composing existing behaviors to form new behaviors; in Lisp’s case, by literally composing lists.
Another (novel?) approach to homoiconicity is to make commands themselves a first-class datatype within the language. A programming language does not need swathes of Python/C-style operators and statements to be fully featured; only commands are actually required.
I did this in my kiwi language: a command is written natively as `foo (arg1, arg2)`, which is represented under the hood as a value of type Command, which is itself composed of a Name, a List of zero or more arguments, and a Scope (lexical binding). You can create a command, you can store it and pass it around, and you can evaluate it by retrieving it from storage within a command evaluation (“Run”) context:
R> store value (foo, show value (“Hello, ”{$input}“!”))
R>
R> input (“Bob”)
# “Bob”
R>
R> {$foo}
Hello, Bob!
Curly braces here indicate tags, which kiwi uses instead of variables to retrieve values from storage. (Tags are first-class values too, literally values describing a substitution to be performed when evaluated.)..
When it comes to homoiconicity, Lisp actually “cheats” a bit. Because it eagerly (“dumbly”) evaluates argument lists, some commands such as conditionals and lambdas end up being implemented as special forms. They might look the same as every other command but their non-standard behaviors are custom-wired into the runtime. (TBH, Lisp is not that good a Lisp.)
Kiwi, like John Shutt’s Kernel, eliminates the need for special forms entirely by one additional change: decoupling command evaluation from argument evaluation. Commands capture their argument lists unevaluated, thunked with their original scope, leaving each argument to be evaluated by the receiving handler as/when/only if necessary. Thus `AND`/`OR`, `if…else…`, `repeat…`, and other “short-circuiting” operators and statements in Python and C are, in kiwi, just ordinary commands.
What’s striking is how much non-essential complexity these two fundamental design choices eliminate from the language’s semantics, as well as from the subsequent implementation. kiwi has just two built-in behaviors: tag substitution and command evaluation. The core language implementation is tiny; maybe 3000LOC for six standard data types, environment, and evaluator. All other behaviors are provided by external handler libraries: even “basics” like math, flow control, storing values, and defining handlers of your own. Had I’d tried to build a Python-like language, I’d still be writing it 10 years on.
There are other advantages too. K&R spends chapters discussing its various operators and flow control statements; and that’s even before it gets to its stdlibs. I once did a book on a Python-like language; hundreds of pages just to cover the built-in behaviors: murder for me, and probably not much better on readers.
In kiwi, the core documentation covering the built-in data types and how to use them, is less than three dozen pages. You can read it all in half an hour. Command handlers are documented separately, each as its own standardized “manpage” (currently auto-generated in CLI and HTML formats), complete with automated indexing and categorization, TOC and search engine. You can look up any language feature if/when/as you need it, either statically or in an interactive shell. Far quicker than spelunking the Python/C docs. A lot nicer than Bash.
Oh, and because all behaviors are library-defined, kiwi can be used as a data-only language a-la JSON just by running a kiwi interpreter without any libraries loaded. Contrast that with JavaScript’s notorious `eval(jsonString)`. It wasn’t created with this use-case in mind either; it just shook out of its design as a nice free bonus. We ended up using it as our preferred data interchange format for external data sources.
Honestly, I didn’t even plumb half the capabilities the language has. (Meta-programming, GUI form auto-generation, IPC-distributable job descriptions…)
..
Mind, kiwi’s a highly specialized DSL and its pure command syntax makes for some awkward reading code when it comes to tasks such as math. For instance, having to write `input (2), + (2)` rather than the much more familiar `2 + 2`, or even `(+ 2 2)`. Alas it’s also proprietary, which is why I can’t link it directly; I use it here because it’s the homoiconic language I’m most familiar with, and because it demonstrates that even a relative dumbass like me can easily implement a sophisticated working language just by eliminating all the syntactic and semantic complexity that other languages put in for no better reason than “that’s how other languages do it”.
More recently, I’ve been working on a general-purpose language that keeps the same underlying “everything is a command” homoiconicity while also allowing commands to be “skinned” with library-defined operator syntax to aid readability. (i.e. Algebraic syntax is the original DSL!) It’s very much a work in progress and may or may not achieve its design goals, but you can get some idea of how it looks here:
https://github.com/hhas/iris-script/blob/f9d9298824d05eccb22...
Partly inspired by Dylan, a Lisp designed to be skinnable with an extensible Pascal-like syntax, and also worth a look for those less familiar with non-Algol languages:
http://www.gwydiondylan.org/books/drm/drm_7.html
And, of course, by Papert’s Logo:
https://www.amazon.com/Mindstorms-Children-Computers-Powerfu...
Except the lisps with good error messages don't really use those things without tagging on the file and line number where they came from. You end up working with "syntax objects" or something instead.
There is no need for ugly, cumbersome syntax objects that destroy Lisp.
Whether or not the recording of source loc info is enabled is controlled by a special variable. It is disabled by default, because you probably don't want that overhead if you're calling the reader for reams of data. Functions like compile-file and load enable it locally.
To take advantage of the info for error reporting in a user defined macro, you can simply do this:
(defmacro mymacro (:form f arg1 arg2 ...) ;; get the form using :form
(when (somehow-no-good arg1) ;; something wrong with arguments
(compile-error f "first argument ~s is no good!" arg1))
...)
You get an error which mentions mymacro, and the location. (foo.lisp:10) mymacro: first argument 42 is no good!EDIT: If anything, I’d define homoiconicity by the ability to write the same program (as EVAL sees it) in a variety of different textual formats: e.g. ‘foo is equivalent to (quote foo) and both programs are indistinguishable to EVAL
What makes CL homoiconic has nothing to do with the textual syntax, but rather the existence of a function READ that takes _some_ textual syntax and produces the code to be evaluated in the form of the language’s ordinary datastructures: EVAL and friends consume a representation of the code that is neither just a blob of bytes nor some exotic AST objects, it’s lists, hash-tables, strings, vectors, and all the types of things programmers manipulate every day.
The implication of this is that intercepting and modifying the code between READ and EVAL doesn’t really require any special knowledge: you have to know the semantics of your language, and how to manipulate its basic datastructures, but you don’t need to understand any special “metaobjects” for representing code.