One important thing we learned is that the filter design is a very important part of the whole thing. We used scipy.signal[1] for filter design. One thing that was very noticeable was that even small IIR filters performed significantly better than FIR.
I thought we put our code online a while ago but looks like we didn't. If you're still interested I might have a look if I can put my stuff online later today. Not that it's all that fancy anyways. The C++ part is pure processing and the python interface just puts a gui around it for filter design and calling the application.
[1] https://docs.scipy.org/doc/scipy-0.15.1/reference/tutorial/s...
It did indeed require a lot of additional tweaks and benchmarking. Honestly don't think any part of our original algorithm or the tricks in the paper made it into the final version. Just doing a Fourier type thing works rarely (~20%), but the principle does work; you can get it working ~90% of the time. That said, I'd be surprised irregular heartbeat were doable in a short recording period. But I'd love to be surprised!
[0] Hands Free Heart Rate Monitor (free, private) https://apps.apple.com/app/id1457939511
Their techniques are at least patent-pending if not fully patented by now, which is going to limit adoption.
Also, tuning parameters is hell.
I don't think he is willing to share the code, but my guess is that a python implementation should work just fine, with very nice results :) (I saw the results applied in his research on videos with impressive results). It took him something like several months to have a code without major bugs.
[1] Hahn, S., Absil, J., Debeir, O., & Metens, T. (2019). Assessment of cardiac‐driven liver movements with filtered harmonic phase image representation, optical flow quantification, and motion amplification. Magnetic resonance in medicine, 81(4), 2788-2798.
Sharing code to the public in research depends on several factors, it can get open sourced when the thesis is done, it may depends on who is paying the research (eg: some industrial PhD are not public), long term plan exist to publish the code but there are only 24h a day and a whole lot of work to do, the author/uni may want to make a business after the thesis, and a lot of other parametres. It gets even worse when the code is used in medical applications as a validation process with all the relevant authorities has to be done. This can take a lot of time and freeze the code in the waiting for a certification. Also sharing code between researchers is very usual, it doesn't mean the code has to go public. In this case I think it is part because the original matlab code is patented, research is ongoing and as it is research code it is not necessarily public ready.
It doesn't mean he will never release it.
If you desperately need it, the matlab code is available, and has the same functionality, you are free to adapt it or use it directly (for research purposes).
I must say I don't like that the original code is patented, but I see hundreds of reasons why the authors patented it even if I don't particularly agree :)
More notes and code here: http://nhergert.ozeo.org/doku.php?id=projects:remoteppg
The pipeline started with face detection, then found the mean colour value of a rectangle of forehead. It then did some DSP and returned the pulse frequency.
It worked, but only when the person held extremely still very close to the camera, with auto exposure turned off.
Disappointed with how badly mine worked, I took a look at some open source webcam PPG projects to find out how they were working. It turns out they were pretty similar, except most of them were aggressively filtering the pulse signal to be in the correct range, without really checking there was already a signal there. Of course by taking white noise signal and filtering it to a certain frequency range, you'll get a signal in that frequency range.
I now believe that to make video PPG work, you need some of (1) a very good camera, i.e. not just a low end webcam (2) quite a long recording, i.e. 10s-1m (3) a controlled environment
The guy who used it seems to imply his version had a bit of 'special sauce'
Even in little videos on Instagram it could be a cool and eye-catching effect.
I was really excited about it, because, as a cardiologist, I immediately thought we could use this to identify irregular heart beats in the patient waiting room, for example. This was pre Apple watch etc. I was thinking people could briefly sit in the chair in front of the camera if they are willing to be recorded, as a quick screening whilst they wait for the doc.
Unfortunately, I found it incredibly difficult to reproduce their results, even using similar data. I remember videoing my foot as I have a relatively prominent posterior tibial pulse, which in good light I can actually make out visually (but not in the video I took, as intended). The Eulerian magnification didn't seem to do anything.
Has anyone else had better luck? I did wonder if maybe I had to use specific video capture equipment, certain frame rates or filters, but if so that wasn't clear from the documentation?
It's been 8 years since this paper came out, and yet I still haven't seen it being used in the 'real world', so I am slightly suspicious that I am not alone in my experiences.