Anyone executing via lit orders is either forced to do so or an idiot. That’s why most of the market doesn’t execute via lit orders. Which is fine. The trade is still reported ex post facto, and the inefficiencies this creates are always less than the convoluted auction formats one must use to make low latency non-advantageous.
As for retail execution, while on net there's a standard and strong argument that a less regulated market is the most efficient, retail execution is most definitely at the bottom of the totem pole, with an order shopped around to parties that can pick and choose the profitable orders before it is sent to wider liquidity pools. I think that this side of the debate is more about evening out the playing field. On net the market may get less efficient, while slower speed participants have an improved experience. These aren't contradictory.
Wall Street lobbies to ban HFT every few years, hoping people who don’t like how it looks will back them up.
Slower participants in equities are block traders. Folks moving hundreds of millions if not billions at a time. Large institutional investors. Dealers. Retail, on the other hand, would get hosed, though it would also become much more profitable to execute, so maybe that brings some service perks for larger retail traders.
Since this is hackernews, I'd imagine there are a decent amount of people with low float tech stocks. Those things can be impressively squirrely. I've moved a billion dollar company 5% with a $100,000 order by being a klutz.
Granted, these situations are usually in times of market stress, but these are times when for better or for worse people do need to raise personal cash on occasion.
By doing so you have completely identified yourself as non- informed, slow human flow. Ex: if you are looking to sell, it's blindingly obvious that the next likely move from you will be to lower the asking price. Even human traders will be a able to take advantage of that situation as a bread and butter trade.
One important aspect is that a lot of this is in terms of opportunity cost and risk. If you are posting the order at a "bad" time (let's say market makers are not long inventory and looking for liquidity), that's when one should expect front-running style action, as they want liquidity ahead of you. Likewise, if you're hanging out there and a market blip in the sector or in the depths of the market complex moves against you, you will get filled and "miss out" on the higher price that the price will settle at. And while you may think they don't care about a 50k order, these robots are hyper optimized and will have had PhDs and 9 figure plus data and infrastructure costs explicitly designed to capture every cent. That's why it's so obnoxious... It's not just market makers either. I know of a prop shop trade that involves harvesting rebates on trending stocks (stuffing the ask and amplifying the trend while receiving credits... if you've looked at stock charts you may have seen a seesaw pattern of liquidity exploration), and if you step into an active trade like that no doubt there will be at least some basic conditional logic to take advantage of stale liquidity.
I walk my very non-tech mother through manual executions on occasion. She finds it very funny that I can see her order, and without prompting has commented about how annoying the little game is.
Numbers - Let's say it's a 50 dollar stock that doesn't get a ton of volume. Most stocks are surprisingly illiquid. Which makes sense because of course nobody wants to deal with HFTs. The lit order book is almost a reference price for the actual trading that happens behind the scenes (midfills at dark venues, etc). Wouldn't surprise me at all to see 10 cents of additional slippage. That's $100. Also wouldn't shock me to see more if it's a smaller stock. Of course it's also fairly common for there to be midpoint liquidity right there for you to take. It just depends on the positioning of each of the participants, and a retail trader is at a distinct information disadvantage.
That said, it's highly unlikely that your 401k is at a DMA (direct market access) broker. Your order is probably first going to go to an internalizer (crossed with other customers), and then flashed to prop firms who will have the ability to take your order (and if they do it probably technically means that you've missed some money somewhere, although it may be in any number of obscure areas), and finally you're going to get sent to the market and pools of liquidity via a decent execution engine. On net, these routes don't work out that badly for retail participants.
Also, if you're talking a highly liquid ETF like S&P or Qs, don't worry about it. Just hit the bid.
That said, I would recommend upgrading if you can. Use a midpoint order type, split your order into chunks, spread it out time-wise a bit. Market On Open and Market On Close order types are also widely available at better retail brokers. I think that these are the most fair fills you can get. Split it 50/50 between open and closing auction. It's just a drop-down order type selector, and you can queue for the auction when you set up the order (say, early morning before the day starts) and walk away for the day and come back to filled orders.
Don't use market orders outside of the huge indexes and megacaps. You're guaranteed a bad fill, and then you also run the tiny risk of a truly awful fill (if something happens machine speed before you can blink... been there done that).
Market On Close / Market On Open orders are really easy to use. Brokers like Schwab and Fidelity and interactive brokers will support them. More people should use them. You'll be getting fair fills side by side with smart money.
There is a caveat though, which is that top-of-book liquidity is increasingly thin every year. It doesn't take that much size to hit the bid, take out the first thin onion layer of liquidity, and have the spread widen away from you. If you look at the live order book depth you will see that the top of book is often thin and flittering. The deeper liquidity will react to the top levels getting cleared before you can blink. (That's why if you have a non-small order and want the bid price, sweep the bid and go a few cents under, you will get a much more reliable fill and won't be left hanging with the liquidity instantly repositioned a sub-penny below you).
Because lit orders get front run. Every sophisticated participant/algo is exceptionally efficient at extracting money from less sophisticated participants.
As someone who trades decent volume but doesn't have a fully institutional grade workflow, I have the fortune of dealing with this...
Simple lit orders (posting an order directly to an exchange) will be taking advantage of by both market makers, by HFTs, and by smarter execution algorithms. The algorithms running the bids and asks will widen spreads. Sell orders will peg to one cent below your ask, and if flows start to reverse, they will pull their liquidity and the slower participants get their liquidity swept through (adverse selection).
The next step up is to use something like a midpoint algorithm or hidden order, but hidden orders will be pinged with one share from the robots and you will get sniffed out and positioned against. If they detect size in a midpoint algorithm, the liquidity in the opposite direction will evaporate, and they will "walk" the dumb midpoint algorithm down, take the liquidity, and then reset the mid back to where it was. The list goes on. It's generally an awful environment for "regular" participants.
Moving on from simple improvements available to the more advanced retail space like midpoint algorithms and VWAP algorithms, you have algo routes that are explicitly designed to take advantage of the "lesser" order types. If they are in a position to get a fair fill, they will rest the order in case they see a situation they can take advantage of, and only take mid fill if the outlook deteriorates (this is all millisecond time frame stuff, but the orders will be worked in an automated fashion throughout the day - time frame is configurable).
On the more developed institutional side, liquidity is sourced in dark venues designed to ward off HFTs and front-running, or sourced in fair flash-auctions which are again designed to ward off hfts and information leakage from the auction spawner.
So the argument would be that perhaps the modern developments like batched flash auctions should just be the new baseline, and designed so that all of the participants feeding into them get an equivalent quality of fill.
These "phenomena" are fairly significant. Let's say you have a 100k position in a smaller cap stock. You may move the stock down a few percent if you start walking down your order and it becomes clear that you are looking to take liquidity. Vs 100k in one of the more advanced order routes where you're basically going to get filled near mid. And of course it goes without saying that 100k won't even move the needle in the institutional routes.
For a while I got so sick of it that if I was looking to buy back my short options (the same things happen in the option space, but with more slippage), I would stuff a basic midpoint algorithm on the underlying, it would be sniffed out and liquidity would evaporate, price would fall, and I'd slam the ask to buy/cover my short calls on the price drop. At least I could get a fair fill when I played two different areas of the market complex against each other... It's just a pain. To the average participant, they will find that liquidity is there when it suits the counterparty, yet not there when they need it.
NBBO/best bid offer itself can be illusory. There are many situations where if you sweep the bid, you will get a fair fill, but I'd you just hit the bid price, you will essentially take off the very small front order of an iceberg order, they will run their calculations, and the liquidity pegs a cent below you if it suits them. That's how it works.
This goes for all areas of the financial market, including the bond market itself, and it contributes to systemic fragility in addition to harvesting retail money.
Granted, almost no retail participant is actually shipping orders directly to exchanges like I laid out. They are going to payment for order flow routes. These are actually fairly efficient, but again, remember that if you are posting a bid or ask, exchanges pay you (yes, you actually net money, albeit small) to post these orders, and anyone feeding into PFOF routes is getting this income taken from them. The frontrunning risk in the payment for order flow routes is also much more severe, since your order is getting blasted out in all directions before it is posted. So when those sorts of routes go wrong for retail traders (ex making the mistake of posting a large order during a major market event), they're could catastrophically get screwed.
It's also worth noting that retail does have access to a relatively Fair auction system though. Open and closing auctions are probably the best ways to fill orders. Just be careful not to ship too much size into them since a large enough net imbalance (say in a small cap stock) in a closing auction will have the same "walk down the price" effect that happens with midpoint orders.
Personally I think that the institutional flash auctions are pretty neat. For my understanding this sort of liquidity sourcing is growing. I would think that this sort of functionality could be regulated and integrated into the base level market venues.