It's worth it. And yes, I have done that, it's not a mystery, they all provide a API/ABI for interacting with them[1], such as OpenSSL's Engine API or PCKS#11.

Edit: There's even a project to literally do this directly from PHP[3]

[1] http://stackoverflow.com/questions/10796485/interfacing-with...

[2] http://en.wikipedia.org/wiki/PKCS_11

[3] http://stackoverflow.com/questions/3231293/how-to-interface-...