There certainly were doctors and scientists who saw the harm of smoking prior to the general consensus. It seems as though this consensus is a result of both mounting medical evidence and a fundemental shift in the way medicine approaches disease, with the field of epidimiology expanding from the study of infectious disease to include a variety of chronic disease and cancers. IIRC, I've read references and anecdotes of smoking and tobacco having obvious negative health impacts as far back as the US Civil War (1860s).

There's a good record in "Research on Smoking and Lung Cancer: A Landmark in the History of Chronic Disease" (The Yale Journal of Biology and Medicine, 1989) [0]:

> By the 1930s, some evidence had been obtained that the incidence of lung cancer among males was increasing. The evidence came from three sources: official mortality statistics, pathologists' reports of autopsy findings, and the observations of physicians who specialized in the treatment of lung disease.

> Speculation about these factors continued, but there was also much criticism of the view that the reported increase in lung cancer was credible. . . . Factors which were listed as likely to be responsible for an artificial increase were better diagnosis of the disease and increased longevity of the population.

[0]: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2589239/pdf/yjb...

Those would be the "conspiracy theorists".