The documentation is less incoherent than you assume. My own background is in linguistics, so I am familiar with the literature on the history of the Roma that take evidence of historical language contact into account. The relevant chapter in Matras’ Romani: A Linguistic Introduction (Cambridge University Press, 2004) has a pretty accessible overview of prior scholarship and the modern state of the art. As Matras points out, interaction between the Romani and the Byzantine Greeks was ongoing from the eleventh century at the latest, so they had left India well before the Mongol invasion (which happened in the mid-twelfth century).