The method to avoid overfitting was to use the model with the lowest validation loss, not training loss. I was able to change the track around my house with reasonable success. I think would need many more example turns in the training data to become robust.