Maybe, or maybe not. I think there are many different kinds of work all lumped together under the "software engineering" umbrella, and tools which are indispensable for some people can seem unimpressive or unnecessary to others. We're not all spending our time in the same way, working on the same kinds of tasks.

I don't find myself doing huge structural refactoring often enough that it would be worth my while to learn a specialized tool for doing it; I'm a quick typist, and it's easy enough to just blat through the changes by hand on the rare occasion such a thing is necessary. But I don't work on giant line-of-business apps or sprawling web services, and I can see someone who did a lot of that sort of work having a different take on the matter.

> I guess it's hard for me to edit things that I don't see right in front of me or aren't super simple changes (like name changes). Or at least, basic things I can reason about (such as finding by regex then deleting by textobject or something).

This is actually what's nice about tools like ast-grep. The pattern language reads almost like the code itself so you can see the transformation right in front of you (at least for small-scale cases) and reason about it. TypeScript examples:

  # convert guard clauses to optional chaining
  ast-grep -pattern '$A && $A.$B' --rewrite '$A?.$B' -lang ts

  # convert self-assignment to nullish coalescing assignment
  ast-grep -pattern '$X = $X ?? $Y' --rewrite '$X ??= $Y' -l ts

  # convert arrow functions to function declarations (need separate patterns for async & for return-type-annotated though)
  ast-grep -pattern 'const $NAME = ($$$PARAMS) => { $$$BODY }' --rewrite 'function $NAME($$$PARAMS) { $$$BODY }' -l ts

  # convert indexOf checks to .includes()
  ast-grep -pattern '$A.indexOf($B) !== -1' --rewrite '$A.includes($B)' -l ts

The $X, $A etc. are metavariables that match any AST node and if the same metavariable appears twice (e.g. $X = $X ?? $Y), it requires both occurrences to bind to the same code so `x = x ?? y` will match but `x = y ?? z` won't. You can do way more sophisticated stuff via yaml rules but those are less visually intuitive.

Sadly coding agents are still pretty bad at writing ast-grep patterns probably due to sparse training data. Hopefully that improves. The tool itself is solid!

Came here to recommend ast-grep too!

If your editor of choice supports an extension (vscode does for example) it's a very easy on-ramp for a better search/replace than regex offers. It's syntax-aware so you don't need care about whitespace, indentation etc. Very easy to dip your toes in where a regex would get complex fast, or require multiple passes.

I converted a codebase from commonjs to esm trivially with a few commands right after first installing it. Super useful.

I hope LLMs eventually start operating at this level rather than raw text. And likewise for them to leverage the language server to take advantage of built in refactorings etc

I don't know which editors support this but there's also, for lack of a better word, context aware grep. For example, search and replace foo with bar but only inside strings, or only inside comments, or only variables, or only methods, not class names (maybe ast-grep does this).

While at it, https://github.com/semgrep/semgrep was around for several years, too.

TBH, it's actually not as hard as you think, most of the time, what I do is just select the whole syntax node and delete it, copy it, or replace it, and only 20% of the time would actually require deliberate understanding of how the AST is structured in the current language I'm coding in.

I feel the same way. Learning Elixir and getting into macro writing was really helpful for me. Lisp is also the same way though for me the syntax of Elixir resonated a lot better for me. Many paths to the destination and all that, but figured I'd share in case it's helpful to someone