I live and work in the Tokyo area, and my anecdata are similar to yours. Of the hundreds of people I know through work or private life, exactly two ever mentioned looking forward to the games, most seemed uninterested and passively resigned, and a few have been actively opposing them. The percentages would probably break down somewhat differently, though, if I knew more people in the travel industry or people whose primary information feed comes from the mainstream media, which were enthusiastic about the games up until COVID.