Yes, you use it to select the parameters of you optimiser, like step size and momentum schedules and other tuneable things. Look up MAML (meta abstract machine learning?)

11 years ago, I did something like that. I was trying to use particle swarm optimization to efficiently update neural network weights, it didn’t work well.

Don’t ask me why I even tried, I was a dumb but code happy undergrad!