First, having 'infinite compute' is a way for researchers to be sure that compute isn't the thing holding back their method. So, DeepMind made a protein folder using all the compute they had available; later others managed to greatly reduce the amount of compute needed to get equivalent results, by re-implementing and innovating on DeepMind's initial write-up.

Second, I think there's a lot of interesting ground to explore in hybridizing ML with more classical algorithms. The end-to-end deep learning approach gives us models that get the best scores on imagenet classification, but are extremely prone to domain shift problems. An alternative approach is to take a 'classical' algorithm and swap out certain parts of the algorithm with a deep network. Then you should often get something more explainable: The network is explicitly filling in a specific parameter estimation, and the rest of the algorithm gives you some quality guarantees based on the quality of that parameter estimator. I saw some nice work along these lines in sparse coding a couple years ago, for example...