As Dravidians your people have been on the receiving end of caste imposed by Aryans, so unless you were Brahmins they did not have the same incentives to preserve the system once they left the suffocating environment of India.