As far as I've heard from Singaporeans, they didn't "annoy" Malaysia. Ethnic Chinese politicians got popular (well, especially Li Guangyao) and threatened the stability and harmony of majority Muslim Malay country.

As one Chinese taxi driver in Kedah told me, they were supposed to keep quiet and stay away from politics. At least back then, especially influential Malays feared their absolute power is in danger. Didn't help that Li (who always wanted Malaysia to be a state were every ethnicity has equal rights) openly spoke about how poor Malays are held back both by their religious and lay leaders.