they do mean dynamic programming in a more general sense: but dynamic programming was invented originally to solve Bellman's equation. it turns out many other problems have a similar structure which is quite surprising! before bellman's seminal work in 1940s, it wasn't known how to efficiently these problems. indeed, you can often do something like Q-learning for finding approximate solutions to dynamic programming problems even faster.

the general search procedure is just A* search but with the beam search pruning each time you expand the fringe of exploration.

i don't know a good source, i picked this up from a bunch of courses/research.