The Open Market Operations you described (completely different from the Overnight Rate) are a form of stimulus. Because money always seeks higher risk-adjusted returns, the Fed will buy treasuries, which drives down their yield. This makes them an unattractive investment, so the money goes where it can get a better risk-adjusted return. That's usually in the market. So by adjusting treasury note yields, they can stimulate the economy. Furthermore, those treasury bonds are bought with printed money, so this is effectively a way to inject massive amounts (trillions of dollars) of printed money directly into the economy. It's pretty crazy when you think about the power that these policies have.
As a counterfactual: we can imagine a world where the Fed 'sets' these rates by just announcing the rate but not engaging in any transactions and legislators pass a law that forbids banks from borrowing from one another at any other rates.
The interbank lending market would just not clear in that case, and you'd either have a glut of demand or a glut of supply.
The Fed sidesteps this by actually borrowing and lending in the interbank lending market. And that's economically equivalent to buying and selling very short term bonds, even if the legal form is different.
> The Fed sets the interest rate directly.
So which interest rate does the Fed set directly, and how does that setting have any effect on the economy?
(I know they have interest on excess reserves. I already accounted for those in my original comment. I know, they are annoying and misguided. I was mostly talking about the system they used before, ie up until about 2008.)