Stop communicating with him via phone or in-person. All of your communications need to be email/SMS/chat, which will provide documentation in case of a legal dispute.

Then talk to a lawyer.