Viral pneumonia is among the most common cases of death among the elderly.
Influenza cases vary significantly between years.
There is a lag of infection to death of two to three weeks.
It would have been invisible for a long time, spreading uncontrolled.
Except that if it really was 'spreading uncontrolled' and giving everyone 'the worst flu', as opposed to a handful of isolated cases which became more than a handful shortly after people started to panic about it, there wouldn't still be more vastly people being hospitalised for respiratory symptoms weeks after measures were taken to halt the spread than in late February. There might not have been much community testing going on then, but hospitals were certainly keeping records of who was coming in with respiratory problems; the well-established fact that comparatively few people were is strong evidence against the hypothesis it had already become widespread by then.
You're also more likely to develop pneumonia from Influenza than from COVID-19, especially if you're young. If you presented to a doctor oblivious of COVID-19 (or SARS), they would assume it is Influenza.
They're still higher than usual, and so any 'even at the peak' appeal to curves acknowledges that epidemiologists were probably correct and numbers of cases rose very rapidly from near-negligible levels, before slowing due to social distancing measures. If the disease was widespread much earlier, excess hospitalisations and deaths during that period ought to be much higher than they are now with aggressive social distancing having been place for nearly two months. It's not like France has limited access to healthcare or was more likely to chalk a surge of hospital inpatients with severe respiratory symptoms off as something else in late Feb or early March than in late April, when despite lockdown they were much higher...