I clean out my evap coils and condenser coils annually, and replace my filter twice a year (for efficiency, not for health). I'm well aware of the dust that my filter removes, which makes my indoor air cleaner than outdoor air. My allergen-sensitive wife proves experimentally that it has a strong positive effect for a couple weeks every spring and fall. And I'm aware of the crud that develops in the condensate tray. There's little crud in my central air system, but when I used a window AC in an apartment and the unit wasn't leveled properly it got pretty bad.

More generally, though, the presence of unattractive substances does not imply a health risk. My yard is, in the most basic sense, an enormous pile of dust, plus a few organics that grow and decay into dust. My home happens to be a few hundred meters from a lovely trout stream, and that stream and its floodplain contain more life from bacteria to algae to fish than any primordial sludge ever had. Neither are dangerous.