The language has ways that the program is expected to do things, such as named let and syntax-rules, but is flexible enough to get out of your way if you have a different idea, such as defmacro.
I wish that the rest of my userland would be as flexible.
You can have a pretty modern userland with Org, Gnus or mu4e, pdf-tools, etc.
I have recently switched to GuixSD after 2 years toying with NixOS. All dirty Systemd plumbing fades away and gets replaced with a minimal service manager written in Scheme, GNU Shepherd. Plus all packages are defined in Scheme. It's very refreshing.
I hope one day we get to see a cleaner kernel, ABIs and compilers. Things have gone a bit out of hand.
Emacs, and to a lesser extent, sch, are a nice first step to making a nice environment. But they still feel ten years old, or more. I use TUIs, and appreciate them. But VLC and my window manager should be beholden to the same level of control and interactivity.
And yes, I am never satisfied tweaking my system to my preferences, because I always hit roadblocks before I get there.
(I am totally fine with the kernel, and compilers and the like using "dirty" code -> that's one of the easiest ways to reach a damn fast system.)
https://www.emacswiki.org/emacs/GuileEmacs
> Guile-based Emacs is a branch of GNU Emacs that replaces Emacs’s own EmacsLisp engine with the Elisp compiler of Guile.
One Emacs Lisp engine replaced by another.
> Lastly, it will become possible to write Emacs extensions in Scheme instead of Elisp
That means Emacs Lisp remains the implementation language and the extension language. Guile provides the base machine and Guile Scheme becomes another extension language.
Somewhen in the future..
Personally I think that project is misguided anyway, since it would be much smoother to use some Common Lisp as an implementation language (Common Lisp and Emacs Lisp both are based on Maclisp). But Common Lisp is not popular with RMS.
> the rest of my userland
One of the reasons I eventually got fed up with Scheme is that interfacing it with the rest of my userland was really quite hard. IO, subprocesses, networking just weren't as smooth as Unix wanted them to be. Everything had to be wrapped in lists before the "big ball of snow" could do magic with it.
Later I discovered Tcl. upvar and uplevel are not syntax-rules or defmacro, but they give you very similar power in practice - I won't say more or less arcane, just different. IO, events and subprocess management are all right there. Best of all, EIAS (everything is a string) makes communicating with the outside world as natural as it should be.
It helps that Tcl got tailcall and lambda shortly after I arrived :).
Makes me wonder if one could port those Tcl features to Scheme / CL via the macro facility, thus having the cake and eating it too?
It is possible to go the other way though ... [1] doesn't go as far as macros, but (provided you don't believe in compile-time/run-time separation) upvar/uplevel could take it there!
No call/cc is possible in Tcl, but coroutines can express more limited forms of continuations. And when Oleg calls out call/cc as a poor primitive, I'm comfortable giving it up :).
Every few years it seems, I become confused. I write some prototypical javascript, which goes well, and I'm puzzled: "Wait, why hasn't javascript become at least smalltalk-like in being alive, integrated, and deeply changable?" Explanations based on developer ignorance pale with the passing years, and with the active tool-building community.
Then I have to mock some browser api, and I remember. Browser javascript - language of a hundred primitives types. A few language primitives, Object, and hundred browser types doing a half-assed job of pretending to be objects. Instead of "get it right, and then write a fast path", browser dev seems to be "write the fast path first, and then pretend we're done". :/
(Yes, Smalltalk has issues scaling complexity - I didn't say scales well.)
Ah well. At least it makes it easy to watch for a language that isn't crippling. Energetic rapid phase transitions are hard to miss.
I think it's really interesting that Lisp developers like everything to be powerful, but only one of these exists:
https://en.wikipedia.org/wiki/Rule_of_least_power
https://en.wikipedia.org/wiki/Rule_of_most_power
Not that I like writing in least-power programming systems (let's say Python here), but it's easier to trust anyone else's programs in them.
The foot-gun the give you is expressiveness, which allows you to make code unintelligible, rather than power to do anything that you feel like. (I'm not aware of any surviving LISP that would allow you to modify the structure of how cons cells are generated, for example).
There's certainly power in expressiveness, but it isn't the godlike power of C, where you can potentially damage hardware.
That, I think, is why we don't have a "Rule of most power".
See for example:
http://www.sbcl.org/manual/index.html#Foreign-Function-Inter...
Previously, even though I used Emacs as an editor, the idea of moving functions that correspond to applications (like mail reading) into an editor looked clunky and even dirty to me.
But I grew disillusioned of the Unix ethos. While it works well for composing very simple programs, it fails to deliver on most other things. For example, Mutt is a great mail user agent. But it is very difficult to customize beyond its author's original intentions using its own little configuration DSL. And scripting it from your shell does not lead anywhere. This happens with many other nice Unix utilities.
In contrast Emacs, understood as a text-mode Lisp machine, is at odds with this approach. The code you write is as first class as the code you run from whatever module you import. Everything is alive, integrated, and can be changed.
As a programmer, I am moving towards more and more abstract application domains for the same reason. Mathematica, PyMC or Scheme allow me to ignore all the horrid details the OP describes and work at a higher level.