> How is it a free speech issue in the other direction when Twitter is not saying anything?

Banning someone or something is them saying something. Endorsement of the debate between different political views doesn’t mean allowing every idea to be acceptable. Letting say the anti vaccination people on your site directly means more disease risk to the general population. However, realizing that doesn’t require you to act. You might decide that’s acceptable where allowing say militant groups planning an insurrection aren’t.

In other words, voluntarily not saying something doesn’t give up your right to free speech.