If I have inside information, and there is a market with 1000 people, and I make $100 by doing insider trading, that money must come from the pockets of those 1000 people. This follows from the fact that the stock market is (for these purposes) a zero sum game.
The error in your argument is that (1) is false. You say "you just call your broker and ask to buy at the market price". But if every market participant was like that, then the order books wouldn't match and the market would break down. There must be some participants who are price elastic. And these are precisely the people whose decisions are effected on the margin, that is, the people who you cause to buy/sell when you engage in insider trading.
So when I buy shares with inside information (say I know the price will rise), I cause the price to rise by a tiny amount, and induce some people to sell who wouldn't have. Since these people wouldn't otherwise have sold, they lose money from my actions.
It is true that insider trading provides information to the market place, and so it is beneficial in that sense. However, it also creates asymmetric information and so harms liquidity. While there is always asymmetric information (I own stocks through my 401k, but I have no idea what MSFT should be worth), insider information is an especially extreme kind, and therefore especially harmful to liquidity.
The SEC isn't intended to protect individual investors, it's intended to protect the market as a whole. That's why they don't care when individual people get screwed, only when people get screwed to an extent that they won't invest in the market. People change their behavior based on what other people do; without full public information, the result of that tendency is to tend towards zero liquidity.
You can order a TV off Amazon and have it shipped to your doorstep tomorrow. That TV was probably assembled in Taiwan from parts made in China and Vietnam, shipped across the Pacific, loaded onto a trailer, sent to an Amazon warehouse, FedEx'd in a jet plane and driven to your house. The majority of firms in that value chain are public companies; what they do is capital intensive. Without functioning capital markets that money would've been used to buy up city real estate and drive up rents.
The argument for has two levels:
1) The trade is a victimless crime. Given the way markets worse, the naïve party would have bought or sold the asset anyway from someone else. Think about how you buy stocks: you just call your broker and ask to buy at the market price. If you do that and end up buying from an insider, you are no worse off.
2) Insiders actually move the price in favor of the naïve party they trade with. If a stock is being sold on the market $29 and the insider knows its true value is $20, they must sell below the other offers (e.g., $28.99) to make the trade. This is in favor of the buying party relative to the outcome without the insider: the buyer will lose $.01 less per share. At large trading volumes, insiders will actually move the market strongly in the direction of the fair price, making all trades of that asset more fair.
Unfair though it may be, society at large does not benefit banning insider trading, and allowing it would make stock market prices more accurate which is actually probably a fairly strong benefit to society.