In Oregon, the votes are received by teams of people that are required to have representatives from each of the major parties. They check signatures, then open the outer envelope and take out the secrecy envelope which has the ballot inside and toss that in another box. The person checking the signature does not do anything else with the ballot itself. Then the pile of ballots in their secrecy envelopes is given to another team and they open the envelope and process the ballot.

Yes, the mailman could commit a federal crime and figure out how you voted for. So if you're worried about that, drop the envelope in the actual ballot box.