The data I found only goes back to 1965 [1], but from the absolute peak in the 70s to today we've reduced per capita use by around 30%. Over the last 20 years we've reduced it by around 15%, going back 10 years and the reduction is only around 5%. Time scale matters a lot here, as does a definition of what "way downdowm" would mean in real % reductions.

I totally agree we've made use of oil more efficient in things like engine efficiency. We've also found more uses for oil byproducts, which can be good or bad depending on your opinion (and again, on your time scale). If we've improved fuel efficiency by 50% and at best reduced fossil fuel per capita by 30%, we've squandered some of those gains by using more oil for other things.

Whether that's a good or bad thing is really up to opinion, goals one has in mind, and how one weighs the trade offs.

[1] https://ourworldindata.org/grapher/fossil-fuels-per-capita