As long as humans are needed to review code, it sounds your role evolves toward prompting and reviewing.

Which is akin to driving a car - the motor vehicle itself doesn’t know where to go. It requires you to prompt via steering and braking etc, and then to review what is happening in response.

That’s not necessarily a bad thing - reviewing code ultimately matters most. As long as what is produced is more often than not correct and legible.. now this is a different issue for which there isn’t a consensus across software engineer’s.

I don't think that reviewing code is so important as reviewing results. Nobody is reviewing the IL or assembly code when they write in higher level languages. It's the end result that matters in most cases.

But we don't evolve IL or assembly code as the system evolves. We regenerate it from scratch every time.

It is therefore not important whether some intermediate version of that low-level code was completely impossible to understand.

It is not so with LLM-written code that's being discussed. More often than not, it does need to be understood and maintained by someone or something.

These days, I mainly focus on two things in LLM code reviews:

1. Making sure unit tests have good coverage of expected behaviours.

2. Making sure the model is making sound architectural decisions, to avoid accumulating tech debt that'll need to be paid back later.

We get stuck reviewing the output assembly when it's broken, and that does happen from time to time. The reason that doesn't happen often is that generation of assembly follows strict rules, which people have tried their best to test. That's not the behavior we're going to get out of a LLM.

Yes, prompts aren't analogous to higher-level code, they're analogous to wizards or something like that which were always rightly viewed with suspicion.

But those are close to deterministic.