I changed the configuration so that my up arrow isn't hijacked. That fixed most of my annoyances. For a quick recall from history that I know is one to four back, I can use the up and down arrows. When wanting to actually search, I will use ctrl + r.

You can toggle back and forth between what is being searched. For example, you can search globally, or one session.