>Always have a naive solution to compare against the AI solution. 'Naive' here may be a simple linear regression, RMSE, or multi armed bandit/Thompson sampling.

This cannot be stressed enough, optimism bias will always push the scientist towards the 'more interesting/complete/new' method and model, but a seasoned practitioner will have the discipline to always establish a baseline (<1 days work).