If you are willing to presuppose the proposition that caste correlates to socioeconomic status (I have no idea), then a sufficiently experienced person probably could. There would probably be slang, mannerisms, or common sayings that would give away your background, just like most Americans of different socioeconomic groups have different behaviors