Re:Writing a Python SQL engine from scratch
by bob1029 on Tuesday January 03, 2023 @02:08PM
(#34236372)
This is very interesting work.
I've been debating writing a SQLite-compatible parser/generator for purposes of manipulating arbitrary queries in AST form.
The biggest reason would be to quickly enumerate statistics about any given query. This would allow for us to run reports about which tables are used from certain areas, how often related tables are joined, etc. This would make refactor decisions substantially easier, since most of our business logic is defined as SQL queries now. Any given install of our product could have well over 10k SQL queries to deal with.
Adding the generator bit would also give us an ability to automatically rewrite queries as needed. This would predominantly be used to standardize the text formatting of any given query, but would also be extremely useful for things like renaming tables or columns.
I've been debating writing a SQLite-compatible parser/generator for purposes of manipulating arbitrary queries in AST form.
The biggest reason would be to quickly enumerate statistics about any given query. This would allow for us to run reports about which tables are used from certain areas, how often related tables are joined, etc. This would make refactor decisions substantially easier, since most of our business logic is defined as SQL queries now. Any given install of our product could have well over 10k SQL queries to deal with.
Adding the generator bit would also give us an ability to automatically rewrite queries as needed. This would predominantly be used to standardize the text formatting of any given query, but would also be extremely useful for things like renaming tables or columns.