This is why you maintain the database seperatly (either in a spreadsheet, or vim, or sqlite, whatever you like) then wrap the LLM with python doing a completion on each row.

Of course everyone is using hosted services and you have to pay to use them that way.