Similar experience here. My PhD research was in a niche (but unhyped) field where it was easy to pump out low-impact papers. My peers who went into more hyped fields had much more citations and much better job outcomes, whether in academia or in industry.

If you want to do high-impact research in the long run, you need to have a strong foundation, which means a solid bed of high-impact papers to point to when you need funding, opportunities, etc.