Homoiconicity isn't required for that, all you need is to be able to map between source and AST enough to identify expression boundaries.

Plenty of development environments for non-homoiconic languages provide the ability to evaluate a selected expression (not just the last one) in a linked REPL on demand.

I love the Lisp family, but I don't know why its advocates often sound like they haven't seen a dev environment for a non-Lisp language since the early 1980s when pointing to “unique” advantages of Lisps.

Homoiconicity is an ill-defined concept. If you take every subjective feature from it, all it implies is that you are able to cut your code into well defined AST structures (what all languages have, but for some it's easier than for others).

So, yeah, anything people do with Lisp could be done for any other language, it only requires more complex tooling. How much more complexity depending on the language, and it varies from "barely perceptibly more" into "that's completely not practical".

To me, "only" and "complex" shouldn't be used in the same sentence. Big part of the job of a developer is to reduce complexity, and we barely do it (depending of the system). What is "barely perceptible" now can be "completely not practical" 2 years down the road.

It's a well-defined concept in theory but in practice it is not:

https://www.expressionsofchange.org/dont-say-homoiconic/

https://news.ycombinator.com/item?id=20657798

Maybe it's not required. However, I can confidently say Clojure in emacs feels like an entirely different development experience than my daily Python work in PyCharm/VSCode and C# in VS. I recommend trying it out if you haven't already.

And I recommend trying out Smalltalk.

As a hobby I want to learn a lisp. Where should I begin? :)

MAL. AKA Make A Lisp. Instructions to write a tail recursion lisp interpreter in a weekend in almost any language. It teaches almost anything you want to know about lisp, plus some practical compiler/interpreter knowledge.

Install sbcl, slime, gnu emacs in some form.

See the book Practical Common Lisp for an intro: https://gigamonkeys.com/book/

If you want to see how Lisp environments used to be on the hey day of Lisp, check the community editions from Lisp Works and Allegro Common Lisp.

Scheme is a nice start. You can write a basic interpreter for it over the course of a few weeks

There is a significant, qualitative difference between using a language that designed for REPL use and one that isn’t.

Those boundaries you talk of are the crux of the issue. A highly dynamic, completely expression based language is going to enable a much different experience. Homoiconicity also plays an important role here, because you can ispect and parse code within the language, with the same functions and algorithms as everything else.

Indeed; I do this every day with Vim and its term command, with Julia, Python, bash, Elixir—it works with any language with a REPL of any kind.

I am not familiar with how Julia and Elixir work, but form my experience Python’s REPL is not the same at all as REPL in Clojure or other lisps.

In Clojure, you literally built your program in memory while you’re writing the source code file, updating the memory representation of your running program part-by-part without ever stopping it.

This experience is absolutely magical, and it’s very hard to go back to edit-compile-run loop in other languages after that. It is completely different from “my language has a REPL and I can execute parts of code in it”.

No, Julia and Elixir are the same as Python in that regard (Python is more annoying because of the indentation problem). I was just talking about interactive evaluation of expressions selected in the editor. Although I’ve messed with Clojure a bit, I admit I don’t fully appreciate what you’re describing. I’m a little afraid to find out, frankly, because it sounds seductive.

Julia is much closer to Clojure than to Python in this regard, which brings back the point from above that homoiconity isn't the key ingredient

How so? In julia, one problem is that you can’t redefine datatypes in the REPL. You need to restart.

Yes, that restart is painful exactly because it interrupts the "built your program in memory while you’re writing the source code file" OP talks about.

my inkling is that the more special forms the more difficult it is to repl a language. if everything is an expression not only can you eval parts of it, but that is enough to support quite a bit of extensibility sans macro.

a comment here some time ago explained nicely what most of these "unique" advantages of homoicinicity are actually about; it was essentially another step on the same spectrum of statement-languages and expression-languages.

in particular this does not mean that it is better but the same advantages of being able to put anything into anything that you find in expression languages like rust in lispy languages are also syntactically in editing source code.

since I started using shrink/grow selection in vscode I have much increased interest for simple syntax aware editing, something simple would be moving blocks of code token by token

Not only that, Smalltalk, Mesa and Mesa/Cedar workstations, all shared the same interactivity as Interlisp-D at Xerox PARC.

The homoiconicity of Lisp is not what gives it this superpower. It is rather Lisp's dynamic nature that actually allows you to keep running your project while also developing it that is crucial - something that Clojure running on the JVM can actually only approximate.

For example, in a full CL, you can modify a class that is currently being used, and objects of that class will adjust accordingly (for most modifications). This is not possible for the JVM - to modify a class, you would have to unload it, which can only happen if the Class object is garbage collected, which requires all instances of the class to be GCd, and the ClassLoader that loaded that class as well - only then could you add a field with a default value to the class.

i get the magical aspect of repls in scheme and CL. not in clojure. not sure why. so i don't like clojure, but i feel like i should ?? why is that?

As I said, I think Clojure is significantly iimpacted in how much dynamic reconfiguration it can handle by compiling down to the JVM.

Still, personally I have little to no experience with Clojure, so I can't really comment too deeply.

i don't think that's it. i like scheme, and clojure is not scheme. it may just be that.

When you evaluate a form deep in a function, how do you handle variables that are defined "outside"? How can you evaluate such expressions?

Cider (the clojure plugin for emacs) has a variant that prompts you for the the values for those variables.

Which function is that? I couldn't find it in the docs

hmmm ... what if it's a closure, and it depends on a huge lexical environment?

Oh nice. That's one place where Clojure beats Common Lisp then..

Does it? Asking for a value when encountering an unbound variable is a default restart

    $ sbcl
    This is SBCL 2.1.1.52.HEAD.321-f8a57bcca, an         
    implementation of ANSI Common Lisp.
    More information about SBCL is available at 
    <http://www.sbcl.org/>.

    SBCL is free software, provided as is, with absolutely no warranty.
    It is mostly in the public domain; some portions are provided under
    BSD-style licenses.  See the CREDITS and COPYING files in the
    distribution for more information.
    \* (\* x x)

    debugger invoked on a UNBOUND-VARIABLE in thread
    #<THREAD "main thread" RUNNING {1001860103}>:
      The variable X is unbound.

    Type HELP for debugger help, or (SB-EXT:EXIT) to exit from SBCL.

    restarts (invokable by number or by possibly-abbreviated name):
      0: [CONTINUE   ] Retry using X.
      1: [USE-VALUE  ] Use specified value.
      2: [STORE-VALUE] Set specified value and use it.
      3: [ABORT      ] Exit debugger, returning to top level.

    (SB-INT:SIMPLE-EVAL-IN-LEXENV X #<NULL-LEXENV>)

0] 2

    Enter a form to be evaluated: 3
    9
    \*

Yeah, in fact, Clojure needs special tooling for this, while it’s built into CL’s execution model.

This is interesting tool[1] that allows much the same with Clojure

[1]: https://github.com/IGJoshua/farolero

Actually, yes I seem to remember it doing that a while ago. I think my Emacs setup has messed up somehow. Now I just get:

   Debugger entered--Lisp error: (void-variable k)

And then a stack trace. No option for restarts.

It works if I run from the command line though..

I suspect you are right. Rebinding an inner function inside another function that contains variables probably will not work. But you could still alter and rebind the entire thing. And that is still flexible; you can still change a running program on the fly with only that ability. And you can always reference objects at the top level.

Cursive has a repl powered debugger just put your breakpoint there use the repl to trigger the call then use intellji's debugger to step through

If you want to run expressions in context just use the expression editor like a normal repl

Something I haven't tried yet is redefining functions at debug time can't see why it wouldn't work though

Smalltalk famously let you code inside the debugger. You can run code with methods that don't exist yet and add them in the debugger when it notices they are missing.

You can use a REPL powered debugger. You can interact with the debugger to a higher degree than with most mainstream languages. It’s not quite as powerful as CL though, from what I’ve read.

Or you pull out expressions and test them in isolation with given (assumed) inputs.

You can essentially give values to global variables with same name as those outside variables.

By evaluating outer forms first.

The repl maintains an environment.

This is the draw for Jupyter Notebooks as well. Execute code blocks. I remember when I learned this in Matlab.

REPL are super nice for this!

I've had a hard time in Java, Fortran, etc. in determining the right development patterns that are used in this space to be productive.

Modern Java (11+) has jshell, which has interactive goodies like tab completion, documentation display, paren/bracket matching, etc.

it's something.

Step1: Write a test.

Step2, IDE: Right click + Run.

Step2, CLI: $ run <test name>

Bonus: The test is now part of your automated test suite, and will be ran many times to ensure things don't break.

What if you don't know what you want it to do yet? Maybe you have a new API or library you're working with and the documentation is lacking. Maybe you're exploring some dataset trying to figure out what kind of information can be gleaned from it.

Maybe it takes 3 minutes to get it into the state where it failed, so instead of starting over every change you just want to modify the one function over and over.

Doing it in a repl like jupyter has you run the actual code many times as you're writing it so ensure things don't break before you ever save the file. Of course thats not an excuse to skip tests, but if I was going to pick one, I would choose running the code as I write it, instead of testing it after the fact.

A. Then don't submit the exploratory test in your next PR.

B. Then memoize the expensive function to disk. Bonus: Can keep multiple versions on disk, thus can tweak the expensive function while having fast reloads with guaranteed correct version.

Now try to fix the code from the unit test when it breaks and redo the unit test, from the point where it broke.

So that you can run your unit test from some unknown state that may not even be reachable from the current text of the code, rather than running it from the start in a reproducible way? Why would that be something you want?

Unit testing was born in the Smalltalk community, good tooling is orthogonal to having tests.

Throw a breakpoint in the unit test.

It won't work on the CLI version, and unless you are using an IDE with a backtracking debugging it won't help.

> In emacs, for instance, you often use `eval-last-sexp`, by default bound to C-x C-e. This lets you move your cursor to a particular point in the file, often deep in a function, and get the results of just the form(s) you're pointing at.

but that depends on how the environment is looking at the time.

during development, in scheme, i just have a function (reset) that reloads all the files of an app. sometimes i create a thread and poll the filesystem and reload everything that changed in the last second. that way i never have to C-x anything. if i need to eval a subexp i just jump on the repl and do it.

Python can hot-reload an application from scratch, too. The idea here is to not reload everything from scratch, but to redefine functions incrementally/iteratively as you work.

if i change anything in filters.scm then i (load "filters.scm") instead of just sending a snippet to the interpreter to eval. i have this happen when i save the file. same thing, except i never have to worry about sending code or a mis-match between editor and repl. it also forces you to be more organized with code so that you don't override global state. i just code, save, code, save. you can have hooks run things after every save etc ...

Well, a UNIX REPL (read sh, bash, fish, etc) is pretty much "homoiconic", since everything is just a text stream. Of course the value of ARGV[0] will be some executable, written and possibly compiled, but that's no different than any other language which needs to eventually make it to hardware instructions.

You can do this in IDEA's IDEs too, just place a breakpoint and then you can "evaluate expression" in that context to call functions, read values, etc.

you just have to try to understand the difference. the friction is zero in a proper repl, unlike breakpointing and evaling.

Kotlin and IntelliJ IDEA really are a joy to work with in my opinion, they're very much my primary tools at the moment. I'd never really been exposed to functional programming features until I got into Kotlin and now I'm learning Scheme of all things, it's been a bit of a gateway drug for me.

I remember doing this with emacs and C in the 90s

Nice. I definitely want to give emacs a solid try at some point.

i've used emacs for 20 years. i still spend 80% of my day in it ... and vscode is vastly superior for development.

The thing that bugs me about the IDEs is how slow they are. I'm super productive in them, but I really don't feel like they need to be so slow. I'm not even talking about the indexing which I get.

In fact they weren't always unresponsive, seems Jetbrain's products have regressed in the last year, but mostly Webstorm. I use WS on OSX and Win10, and on both I get all kinds of slowdowns - different projects too. Tried disabling features, plugins, different GC, increased heaps. I may have to fire up J. Mission control at this point.

IDEA and Clion on Win10 are mostly fine.

It would just be cool to have all of these features, but the frontend be a responsive CLI.

try vscode ;) ~ 5*10^6 java code base, i don't notice any slowness. it would be impossible to do this in emacs.

I can do this in Java and C with the right IDE.

i don't love java but no one can argue with the fact that it has fantastic tooling.

properly written java is a joy.