We've done similar thing as the OP, I think it will be 2 months now. Our processes were pretty focused on spotting defects beforehands already: we have standardised tasks defined with ACs following given/when/then format that we can write behaviour tests for, we have automated linting, static analysis, automated tests. Then there's also round of tests: both QA (LLM/human driven) and UAT (human). Since I joined we have added architecture tests to that mix, including static as well as LLM guided that look at how code is built and organised + writing documentation for what is being built / product rules (LLM driven). Looking at the stats - the amount of merged PRs has trippled, the amount of issues is decreasing, PRs time-to-merge went down to roughly a day (I don't attribute this particular metric to usage of LLMs in review process). What I've noticed: LLMs are better than humans in spotting edge cases and reviewing code, given they have established conventions and standards (I saw PRS that I'd approve, but LLM was pedantic in reviewing a comment or class name); they are good in finding connections, ie I've had many cases where an agent built something where there was no guidance in the ACs and the reason was that it found related tickets in Jira mentioning given thing and filled in the functionality. LLMs are "cheap" to run = one can run review process for multiple PRs at the same time and forget while they run; they code defensively requiring developer to intervene - for example: I'm working on functionality where user is assigned to review a thing. Only one user can be assigned to review given thing at a time and there are checks for this. In last 3 PRs LLM is telling me - add locks because if there are multiple people reviewing given thing at the same time then the data will be lost. Yes, if we ever change business rules in this place this can happen. Finally LLMs are semi-consistent; sure sometimes they'll do stuff for shits and giggles following some convention I've not seen anywhere in the codebase, I ask what happened, and it tells me it picked it up from some tangentially related legacy place.

Now the bad points: many times QA catches things that would emerge during planning or if developer would question what they're building or look at their work thinking critically (I find it that people are loosing the ability to question things) - LLMs dont help with preventing this from happening. People are loosing track of domain and product rules - in the places I've worked I was never just a developer, I was always a developer/something (forever to my dismay), so during planning or review I could say, from product perspective: this doesn't make sense, we don't need to build this. This no longer happens - there's no time where I can remember the rules that govern the product; PM often becomes the bottleneck of what the rules should even be. I'm also finding that people do not develop good taste - this piece of code that LLM used? Sure, looks good, go ahead, ship it. Path of least resistance is becoming a norm. Anyhow - I'm old and tired, so sometimes I'm finding the LLM pros are greater than cons