If you're interested in progress and growth, I can say with a high degree of certainty that what you don't want to do is apply to work on a very stable team. These people are always confident that they know what they're doing - even when they agree that the outcomes aren't great. They've had lots of time to practice their rationalizations and that kind of cognitive dissonance can be pretty jarring/aggravating.
By definition, starting your own company or contracting will avoid that situation, but you don't need to take your ball and go home just to get the right kind of environment.
Fifteen years later I sit in meetings telling people (some of them still older than me) not to touch the stove because it will hurt, and I have to patiently wait until someone gets hurt before we can discuss the proverbial oven mitts.
The problem with doing something right the first time is nobody appreciates how hard it was.
Sometimes if you do put in quality and do it right in development it costs you in time and then by perception.
When you hit the ship date over finishing the iteration/version/product you have problems after ship from external customers, when you do it right and handle the problem in development before ship, you take a hit in perception internally.
The problems after ship can harm a product/company more than a slight delay in shipping, but today it seems people don't care as much as there is more specialization and larger teams where it is 'not my problem'. Some clients/project managers see a bad product shipped on time as good, and a good product shipped late as always bad. The external perception should play more into that perception not just hitting their part of the project goals/milestones.
Hitting dates is hugely important, but messing up a product with the customer can be deeply problematic. Noone truly remembers a late product after it ships and is quality, creative and functional, they remember a bad product.
I prefer the Valve Time [1] philosophy, get the product right over the date, and don't set dates until you have the product actually ready.
Does a development agency that focuses on this model exist? Of course they should be paid more per hour (or other unit) than your average, less optimized agency, but the sum total should be less than hiring inefficient agencies.
(P.S. I understand the best developers probably don't want to start or work at development agencies, but assuming there are some.)
If you know of a development agency, odds are that it is probably a body shop.
The places that do work like this are more like Unicorns. And Unicorns are damned hard to find and hire -- or get hired by them.
This sort of thing is how I now pay the bills.
In many cases, what somebody does (and why) is not documented, or if it is the documentation is most likely out of date and inaccurate.
To effectively automate a task, you need to fully understand what needs to be done as well as have the skills to do the actual automation. Getting somebody in who is unfamiliar with the job, can work, but in most cases you will end up with an off the shelf system that has been customised to perform the original task but come with a large overhead of maintenance that may take more work than the original task.
My suggestion would be that rather than get an outsider to create the automation tools, up-skill and authorise the staff doing the work now to allow them to gradually automate the process. Make sure you make it worth their whiles to automate.
1) As person with business knowledge, vision, plan, etc, I should fully document the business goal and why we're doing it and what we're trying to achieve at the highest level, then work with the developer to figure out what should be developed to achieve that and where to automate.
2) Don't hire outside person or agency, but bring someone in full-time or as a part-time partner for the long term and incentivize them to get better and better?
Seems to be a good idea to separate out the architecting from the actual implementation. Makes sense, because if someone knows they'll have to be implementing, it could cloud their plan for the ideal solution. Similar to separating out design, UX, and development roles.
I hate to burst GP's bubble, but the attitude and environment at his next job probably won't be any better in regard to "best practices." It's extremely difficult to gauge how your attitude toward development compares with a prospective team's after an hour of chatting. I became a manager last year and quickly discovered that changing others' behavior and habits is next to impossible. And if he does happen to find a new team that matches his work ethic he can be certain that 1 or 2 years down the line it won't any longer.
I think the only two ways to avoid this is to become a contractor and switch gigs every 2 years, or start your own company and hope the people you hire share your attitude.