The main issue is that the thermal camera we're using is only a few hundred pixels tall/wide, so using it for feature detection was rough when we did try that for something. For our purposes, with cameras on a solid mount, the hardcoded transformation matrix is good enough, but I think if you had better cameras and more time/more compute power, something like this could be better.

A bigger issue is that the project is running on a Pi, so just getting the image alignment running along with our face detection wouldve also been a pretty tall task if we want to stay at around ~5fps.