(loop :for x :below 10
:do (print x))
This is a macro that presents a new abstraction/concept that's not "in AST format". It's a concept that's presented in syntax similar to any of the stock imperative languages.That's the raison d'etre of macro: to expose these concepts/abstractions/etc. in a direct manner.
There are bad macros, to be sure, which are half-baked and underdocumented. That's no different a criticism from Python functions being 500 line god functions with boolean input flags, etc. Both cases are cases of bad code.
Python as an example, iteration is a concept: go over a range of things. Complemented with comprehensions that can transform/filter with regular syntax that is always the same (if, f(x) or whatever).
Comapre it to this: https://lispcookbook.github.io/cl-cookbook/iteration.html
Is that worth the trouble, seriously? For sure it is super powerful (Python looping is quite powerful also). But what is the cognitive overhead of that, with custom syntaxes included?
Built-in, of those above, are mapcar and loop.
Python iteration isn't so much about ranges, I thought. See: https://kinsta.com/blog/iterables-in-python/
"You" can provide those syntax abstractions as macros. "You" can also use them yourself. But they then can be embedded in Lisp, you don't need to provide a new programming language version or even a new language & implementation with those abstractions. The user may be sophisticated enough to provide his/her macro abstractions.
I'm curious, when do we actually need that? I've been designing my own PL and one thing I realized early on was that we can either provide high level abstractions to increase the expressiveness of the language for problems that are far away from the machine (e.g., business logic which shouldn't be concerned with how you assign memory in loops), or we can let the users do that on their own.
The second approach is Lispy, but that shifts the responsibility of creating abstractions to users. Users already make mistakes in trivial code; to give them the entire AST is mostly detrimental to them (it's like giving them a bigger gun to shoot themselves in the foot, if that makes sense).
Everyone who reads a for loop in Python quickly groks it. But now imagine presenting it in its AST form... There's a reason much of our code doesn't look like AST—we are not computers; we think in terms of abstractions, concepts, etc. which may be cumbersome to present in AST format.