This is completely true; miners will mine the coin that gives them the best ROI. If Bitcoin's value decreases without an equal decrease in difficulty, the whole thing could crash quite fast.