News for hackers. Stuff that matters.
Re:Just what does “code as data” mean anyway? (2014) by dmitriid on Friday February 16, 2018 @01:48PM (#16394745)
I wish people who downvote my comment would explain what they disagree with.

The entire discussion in this subtree centers around paredit (oooh, it can slice and slurp lists!) and vi commands.

Have you ever ever seen and experienced an actual modern coding environment? IntelliJ IDEA? Visual Studio? Hell, even Visual Studio Code.

The full power of those tools put both vi and paredit to shame when it comes to actually working with code.

Re:Just what does “code as data” mean anyway? (2014) by peatmoss on Friday February 16, 2018 @03:29PM (#16395813)
Since you were replying directly to me, I couldn’t downvote you. But your use of “newsflash” made it come across that you weren’t interested in a civil discussion. I frequently downvote comments that make a point in an unnecessarily hostile or patronizing way. I suspect others do as well.

As for full IDEs, they absolutely have a place. They can even infer things about the structure of your code fairly well. I still find the productivity of being able to work deterministically and consistently with my code at a structual level a pleasure.

I’d appreciate that even in an IDE, if for no other reason than that the structure editing could be done efficiently and leave me some extra system resources for other things :-)

Re:Just what does “code as data” mean anyway? (2014) by didibus on Friday February 16, 2018 @02:05PM (#16394926)
You were downvoted because what you said shows that you do not understand what paredit is.

I'd recommend you read https://en.m.wikipedia.org/wiki/Structure_editor to better grasp the concept.

If you've ever used the XML structure view in eclipse, its the Design tab, you get a better idea. In contrast to its source tab.

Re:Just what does “code as data” mean anyway? (2014) by lazyasciiart on Friday February 16, 2018 @02:34PM (#16395250)
Can you explain it? For instance in IntelliJ I can refactor and have it automatically replace only instances of the word "call" that are actually invocations of the function call(). That's structured editing, not text-based.
Re:Just what does “code as data” mean anyway? (2014) by didibus on Friday February 16, 2018 @02:41PM (#16395339)
Yea, that's true, but your editor pane is a text editor non the less. The refactoring in this case is structured, even more so, its semantically accurate to a specific language. But your editor isn't. A structural editor does not allow you to make edits that would go against the syntax rules of your language.
Re:Just what does “code as data” mean anyway? (2014) by lazyasciiart on Saturday February 17, 2018 @09:00PM (#16403997)
It sounds like you're saying a tool can only be a structural editor if it doesn't do anything else. I want an editor that can operate structurally AND as a text editor - I don't write new code as syntactically complete units in one keystroke.
Re:Just what does “code as data” mean anyway? (2014) by jyounker on Saturday February 17, 2018 @08:49PM (#16403951)
I'd argue that IntelliJ is a structured editor. If you look under the hood the entire system is build around a tree-representation of the code. The "text editor" is a thin varnish on top of that syntactic representation.
Re:Just what does “code as data” mean anyway? (2014) by dmitriid on Friday February 16, 2018 @02:56PM (#16395515)
> your editor pane is a text editor non the less

And the editor pane with paredit somehow becomes a magical structural editing machine?

Paredit knows exactly zero things about your code. The only thing it can do is match brackets/parentheses and move code between them. That's it.

In an actual code editing tool that actually understands code structure and semantics, I can:

- select and move semantically and structurally valid blocks of code

- extract parts of code into a separate variable

- extract parts of code into a separate function

- extract parts of code into a separate module

- safely rename variables and functions across multiple files

- simplify/split/extract/generify code

- ... many more ...

Re:Just what does “code as data” mean anyway? (2014) by dmitriid on Friday February 16, 2018 @02:15PM (#16395042)
Let's see. Here's the original text:

> Rather than thinking in letters, words, lines, or paragraphs, editor modes like paredit let you edit in terms of the structure of your code

So. My question is: which modern programmer thinks in terms of thinks about code in terms of letters, words, lines, or paragraphs?

The truly antiquated tools (like vi and emacs) may still handle code as if it was just text. The actual code editing tools have long been able to deal with the structure of the code. And in ways that paredit may only dream of.

> If you've ever used the XML structure view in eclipse, its the Design tab, you get a better idea.

Erm. The only "think semantically" that paredit pretends it does happens only because Lisp has a rather regular syntax. It doesn't take much thinking or work to move parts of code in and out of parenthesis, or to be able to close a matching bracket.

Actual modern tools know the structure of the code and offer much greater editing and code handling capabilities than that.

Edit:

As a trivial example. Here's IntelliJ and PHP code (yes, PHP). It understands the code, it's structure, and offers context based editing help based code structure and semantics: https://dmitriid.com/i/gm2tmnztha4tenjq.png

Speaking of semantics. Unlike the dumb "move things in out of brackets" the actual tools understand semantics of code. Once again, this is PHP (!), and the tool knows about the semantics of code: https://dmitriid.com/i/gm2tmnbtga4tanjs.png

Editing capabilities available to some other languages would blow your mind.

Re:Just what does “code as data” mean anyway? (2014) by didibus on Friday February 16, 2018 @02:34PM (#16395254)
You were probably downvoted too because you keep assuming other people talking about structural editing have never used IDEs for Java, php, C++, C#, etc.

Nothing is blowing my mind, structural editing is different from what you're talking about, and enables different benefits and also downsides then those.

I think the wikipedia article I linked does a good job at explaining the distinction:

> editors in some integrated development environments parse the source code and generate a parse tree, allowing the same analysis as by a structure editor, but the actual editing of the source code is generally done as raw text.

Re:Just what does “code as data” mean anyway? (2014) by dmitriid on Friday February 16, 2018 @02:37PM (#16395293)
> You were probably downvoted too because you keep assuming other people talking about structural editing have never used IDEs

coming from a person who also wrote this:

> You were downvoted because what you said shows that you do not understand what paredit is. > I'd recommend you read ... to better grasp the concept.

So, people keep assuming that I for some reason have never tried paredit (or written anything in Lisp). So why don't I give you a taste of your medicine.

I suggest that you go and read some documentation on the IDEs I mentioned and try and use them to better grasp the concept.

Paredit is a very dumb tool that only works because Lisp's syntax is regular. There's nothing semantic or structural about the ability to move some words in or out of parentheses or to close matching brackets.

Other languages might not have the same wondrous ability simply because their syntax is more complex. Their tools though clearly allow much better actual structural editing and actual semantic reasoning about code.

Re:Just what does “code as data” mean anyway? (2014) by didibus on Saturday February 17, 2018 @02:52AM (#16399604)
> So, people keep assuming that I for some reason have never tried paredit (or written anything in Lisp)

If you have, its surprising that you believe IntelliJ's PHP editor to be a full structural editor then.

> Paredit is a very dumb tool that only works because Lisp's syntax is regular. There's nothing semantic or structural about the ability to move some words in or out of parentheses or to close matching brackets.

Yes, it is a very dumb tool, but because Lisp's syntax is so simple, it makes implementing a structural editor for it trivial like that. So in return its a benefit. That's why a lot of simple regular syntax languages like XML often have structural editors for them too, because of how easy it is to make one.

> Other languages might not have the same wondrous ability simply because their syntax is more complex. Their tools though clearly allow much better actual structural editing and actual semantic reasoning about code.

I'm not saying that certain IDEs for certain languages don't offer great features which allow edits to be made in ways that are semantically aware and maintain syntactically valid structure. I'm saying that those languages don't offer full on structural editors of their code. So when writing and editing the code, you do so as text, without taking structure into account. Yes, your syntax errors will be highlighted, yes you can perform certain structural refactorings, but the editor is textual and not structural. If you have experienced both, it should be pretty obvious that they feel and are very different.

I don't claim one to be better then the other, I think both are great, but it also depends on the language. If Java had a structural editor it might be more annoying then it'd be helpful. For Lisps it is amazingly useful, more so then what IntelliJ does for PHP. With Lisps, I'd rather have paredit then a background AST which allows me refactorings, auto-complete and error highlighting. But off course, those are fortunately not mutually exclusive and I have those also.

Re:Just what does “code as data” mean anyway? (2014) by xfer on Saturday February 17, 2018 @04:03AM (#16399815)
> Yes, it is a very dumb tool, but because Lisp's syntax is so simple, it makes implementing a structural editor for it trivial like that.

Tell me if it helps you use the loop macro correctly? Paredit only helps you with brace-matching, it doesn't (can't) understand about the semantics of your s-expr and hence not that useful without additional tooling. For other languages the brace matching problem isn't that bad that you need paredit-like tools.

Re:Just what does “code as data” mean anyway? (2014) by didibus on Saturday February 17, 2018 @03:14PM (#16402475)
Correct, structural editing is about structure and not semantics. Paredit does help with the loop macro, it helps you write it in a structurally correct way based on the language's syntax rules.

Structural editors just make sure your code will parse, not that it is semantically correct. It doesn't know the meaning or behaviour of the loop macro, but it does know how to properly structure it so it can be parsed. And so it gives you ways to add, remove and restructure elements of your code such that it will always be valid for your language.

Lisp syntax is really simple, so yes, you don't need much out of your structural editor. Just make sure everything is balanced at all times and allow you to move symbols in and out of parenthesis, and change their order and nesting. That's all you need to guarantee proper structure of Lisp code.

Now, in Lisp, if you used a normal text editor, you'd quickly start to have wrongly balanced forms, you'd be annoyed at how you keep having to shuffle so many things around just because you want to now divide a whole form by 5 or change the order in which two things are performed. Using a structural editor will remove all these pains, and suddenly, its even easier to do these things in Lisp then it is in say Java. Whereas without structural editing, it was way easier in Java.

So Java + text editor = Never had issues structuring and restructuring code.

Lisp + text editor = Omg, I hate all the parenthesis and nesting and prefix notation because it makes structuring code painful.

Lisp + structural editing = Everything I found difficult with a text editor is now not only easy, I find that its even faster and easier to restructure the code then when I use Java with a text editor.

Java + structural editing = I don't know, because Eclipse, Netbeans and Idea do not provide structural editors for Java that I know off. I suspect it would not be as game changer as in Lisp, since java + text editor is already satisfactory.

Everyone who says that structural editing isn't useful, and what is way more useful is code refactoring tools is just missing the point. Code refactoring is useful, but in Lisp, so is structural editing. You could even debate it is more useful.

Different language have different pain points, and so require different tools to address. I agree, I'm not looking for a Java structural editor, but its also hard to go back to editing Java after having mastered structural editing of Lisp code. The same way that its painful to rename a variable in Lisp once you've done it in Java using a good code refactoring tool.

Re:Just what does “code as data” mean anyway? (2014) by dmitriid on Saturday February 17, 2018 @03:48AM (#16399775)
> If you have, its surprising that you believe IntelliJ's PHP editor to be a full structural editor then.

Because, unlike paredit, it actually knows about the structure of my code.

> That's why a lot of simple regular syntax languages like XML often have structural editors for them too, because of how easy it is to make one.

Yeah. It's not "structural". It's just parens-matching and a few very basic actions like "surround with parenthesis". Lispers praise it like a gift from god only because there are no proper tools.

> So when writing and editing the code, you do so as text, without taking structure into account.

I see this bullshit repeated again and again. All paredit does is: select this word, select this thing in matching parentheses, move them around. Somehow this makes it a magical structural editor unlike the "just text" of other editors.

Once again: no programmer in the world thinks or works with code in terms of words and paragraphs. The only thing paredit does is select words between matching symbols. It doesn't make it more structural or semantic than editing PHP in IDEA. It's just a somewhat convenient way of editing text with regular grammar.

Re:Just what does “code as data” mean anyway? (2014) by didibus on Saturday February 17, 2018 @03:57PM (#16402641)
> Because, unlike paredit, it actually knows about the structure of my code.

I feel like you mean semantics here maybe. Anyways, knowing about the structure of code does not make an editor structural. It has to provide editing mechanism that are based around the structure.

Its hard for me to explain, but basically it would be something like if you wouldn't be allowed to type code out of a class or function. You'd need to first add a function, which would always insert a fully valid signature with open and closing bracket, name and all, even if placeholders. So snippets sometimes do that, but they really do it as textual convenience, and its not enforced in any way. You would not be allowed to move half of an assignment by itself, you'd need the full assignment, things like that. So that every edit performed would result in valid syntax after they are performed. Valid syntax, not working code, not code that semantically make sense, just the syntax is valid within the syntax rules.

Most IDEs do not provide this. What they do is they syntax check your text on every text edit. So as you add or remove characters, they re-run a syntax check, and highlight your mistakes. But your edit commands are addCharacter(), removeCharacter(). Its not addFunction(), switchAssignment(), wrapInConditional(), deleteVariable, etc. Some of those are available as commands on top of the text editor, but the editor is still textual, and you can not fully write and edit code structurally. Again, I'll quote the wikipedia article which makes this pretty clear:

""" However, most source code editors are instead text editors with additional features such as syntax highlighting and code folding, rather than structure editors. The editors in some integrated development environments parse the source code and generate a parse tree, allowing the same analysis as by a structure editor, but the actual editing of the source code is generally done as raw text. """

> Yeah. It's not "structural". It's just parens-matching and a few very basic actions like "surround with parenthesis". Lispers praise it like a gift from god only because there are no proper tools.

You're just trying to pick a battle here. I'm making no claim in terms of Lisp vs OtherLanguage. Paredit does meet the criterion for being classified as a structural editor for Lisp code. Yes, Lisp syntax makes meeting the criterion really easy, and paredit is not a complicated piece of engineering marvel, but its a complete structural editor for Lisp code none the less. Eclipse with Java does not meet the criterion. IntelliJ PHP editor does not meet the criterion. This does not mean that Lisp is superior to PHP or Java, or even that it has better tooling. It just means they don't have structural editors, and honestly, they probably don't need one.

In Lisp, the benefit of a structural editor is mind blowing. That's why Lispers praise it like a gift from god. Without it, I would probably dismiss Lisp's syntax as too painful to work with. Yes, it provides the most powerful meta-programming of all other language, but the small things, like messing up your parenthesis balancing, or having to juggle forms in and out of each other is enough to throw off a lot of developers. So paredit is like a godsend, because it completely alleviates those pain points, and turns them into strengths. In that regard, it is unfair to judge Lisp's syntax before you've mastered and used it with paredit or similar structural editors.

> It doesn't make it more structural or semantic than editing PHP in IDEA.

It makes paredit a structural editor for Lisp code, whereas IDEA is not a structural editor for PHP code. That doesn't mean Lisp is superior to PHP, it doesn't mean paredit is superior to IDEA. Structural editor is not a vague abstract concept, its a concrete kind of editor, read the Wikipedia page. If you tell me, hey go use IDEA for PHP and it has a structural editor. And I buy a license, I'd be like, what the hell, this is not a structural editor. Its as simple as that. You can't just redefine what a structural editor is or isn't. They've existed for more then 30 years now, there's a common notion of what it is and what to expect.

> It's just a somewhat convenient way of editing text with regular grammar.

Yes, and that's what a structural editor is. Its when instead of editing text as free form, you edit it with respect to the grammar. I quote the wikipedia page again:

""" structured editors allow the viewing and manipulation of the underlying document in a structured manner """

That's all.

Having said that, yes, it can be confusing what is the difference between that and having an editor which syntax check as the text is edited. That's why there's a mention of this on the wikipedia page which I have now quoted many times.

Now, Eclipse has the UML editor for Java, that is actually a structural editor to some extent, but most Java devs hate it.

Re:Just what does “code as data” mean anyway? (2014) by dmitriid on Saturday February 17, 2018 @04:16PM (#16402743)
> So that every edit performed would result in valid syntax after they are performed. Valid syntax, not working code, not code that semantically make sense, just the syntax is valid within the syntax rules.

As I said. Dumb tool for basic syntax manipulation is elevated to godlike status. After which you end up with dumb statements like these:

- editor modes like paredit let you edit in terms of the structure of your code.

- your editor pane is a text editor none the less.

- the editor is textual and not structural.

- structural editors just make sure your code will parse

and other stuff that means exactly one thing: "our language doesn't have any other/proper tools, so we pretend our parens matcher is the best thing since sliced cheese".

It's just a dumb thing that selects words and matches parens. That's it.

> In that regard, it is unfair to judge Lisp's syntax before you've mastered and used it with paredit or similar structural editors.

Spare me your condescension

Re:Just what does “code as data” mean anyway? (2014) by didibus on Saturday February 17, 2018 @05:25PM (#16403112)
All true and factual statements. It doesn't matter if you judge them dumb or not.

> and other stuff that means exactly one thing: "our language doesn't have any other/proper tools, so we pretend our parens matcher is the best thing since sliced cheese"

You're operating in the hive mind mentality of us vs them. I'm talking objectively about syntax families and types of editors. You'll probably keep being downvoted as long as you continue that kind of discourse on HN.

Given Lisp syntax, a structural editor adds tremendous value. Most users of Lisp syntax languages find it an invaluable tool, and would not trade it in for code refactoring tools or linters. Its okay if you disagree. Depending on the Lisp language you pick, you can also have code refactoring tools and linters if you wish. Or maybe you just don't like the features of Lisp, that's okay too, use PHP. The fact the tool is simple and easy to implement does not change its value add. Think of how awesome the invention of the wheel is, even though its quite primitive.

So at this point, its hard for me to understand your disagreement. Idea's PHP editor is not a structural editor. This is not a criticism of PHP, or Idea's editor. Emacs offers structural editing of Lisp code. I personally dislike Emacs. That doesn't change the fact it supports structural editing of Lisp code.

If you are curious, for example, given Clojure as the Lisp syntax language, Emacs does offer code refactoring and linting as well as structural editing. You can extract functions, auto-complete imports, rename variables, fold code blocks, have syntax errors highlighted, have certain code errors reported on the fly, jump to definition, see source and documentation, find all usage, auto-complete, snippets, auto-format, continuously run tests, debugging with breakpoints, etc.

I personally don't use Emacs though, because I like mouse support and modern GUIs. So for Clojure, I prefer IntelliJ Cursive which has all those features also, and Eclipse CounterClockWise which has most of them, minus refactoring. So I'm just pointing out that even given a Lisp which has all the linting and refactoring features you talk about, how structural editing is still highly valued and one of the best feature of all of those when wanting to code with the Lisp syntax.

Re:Just what does “code as data” mean anyway? (2014) by lispm on Sunday February 18, 2018 @03:08AM (#16405009)
> If you are curious, for example, given Clojure as the Lisp syntax language, Emacs does offer code refactoring and linting as well as structural editing. You can extract functions, auto-complete imports, rename variables, fold code blocks, have syntax errors highlighted, have certain code errors reported on the fly, jump to definition, see source and documentation, find all usage, auto-complete, snippets, auto-format, continuously run tests, debugging with breakpoints, etc.

Basically the stuff Lisp IDEs had in the 70s.

Re:Just what does “code as data” mean anyway? (2014) by didibus on Sunday February 18, 2018 @06:34AM (#16405563)
I'm unfortunatly not old enough to know. But I've heard a lot of praise for the era of Lisp machines. It's unfortunate that most of that legacy has been lost. Common Lisp doesn't actually offer that much in terms of tooling, at least, in the free department that I've tried.
Re:Just what does “code as data” mean anyway? (2014) by lispm on Sunday February 18, 2018 @07:49AM (#16405826)
Pre-Lisp-Machine IDEs were already more advanced. See for example BBN/Xerox' Interlisp mid 70s.

Generally Lisp still offers a lot, but if you don't know what to look for, you might not see it even if it is before your eyes. Even though the Clojure community reimplemented much of SLIME.

It also might not be important to you. Stuff like having a Lisp compiler written in Lisp, having an actual interpreter, being able to dump images, low startup times by default, break loops, readable stacktraces, type checking compilers, resumable error handling, embedding in C applications, whole-program compilers, compilers which can create C code, ...

Re:Just what does “code as data” mean anyway? (2014) by khedoros1 on Friday February 16, 2018 @07:13PM (#16397876)
I downvoted because your comment sounds argumentative and condescending. So does this one.