News for hackers. Stuff that matters.
Re:Abstract Syntax Tree for Patching Code and Assessing Code... by heinrichhartman on Monday June 07, 2021 @02:53AM (#27419795)
For me, this is the critical bit here:

> One of the problems of patching code using ast package of Python is that it loses all the formatting and comments of the original source code. This can be solved by making the patch script a little smarter. Instead of having it unparse the entire patched AST and write that to disk, we can make it unparse only the nodes it modified and insert the modified code at the corresponding line number in the file. The ast nodes have lineno attribute that can be used to retrieve the line number of the file to be injected the patched code with.

Parsing & Patching the AST is relatively easy, with the provided tooling. I always struggled to write the modifications out in a minimal way. Looks like line-number information does get you over this hump.

EDIT: Looks like the python community has already a package for this: https://github.com/PyCQA/redbaron

Re:Abstract Syntax Tree for Patching Code and Assessing Code... by BiteCode_dev on Monday June 07, 2021 @07:48AM (#27421441)
For preserving comments, use redbaron: https://github.com/PyCQA/redbaron

It's lossless, besides, the API is much nicer.

Re:Abstract Syntax Tree for Patching Code and Assessing Code... by shpx on Monday June 07, 2021 @07:19AM (#27421250)
Autoformatting the code with Black before and after unparsing is how I avoided this problem.
Re:Abstract Syntax Tree for Patching Code and Assessing Code... by trankas on Monday June 07, 2021 @07:48AM (#27421439)
Never used it but it appears that Facebook/Instagram have a format preserving CST library for Python: https://github.com/Instagram/LibCST
Re:Abstract Syntax Tree for Patching Code and Assessing Code... by megous on Monday June 07, 2021 @08:07AM (#27421579)
I wrote a tool for refactoring JS a few years ago, and the way I solved this was to not modify the AST and regenerate the code from it, but only to use position information from it, and patch source code directly based on that positional information.