If you did, it wouldn’t really be a big deal. Toddlers daily encounter playgrounds full of tiny rocks, wood chips, etc. which are more dangerous than the BBs. Swallowing BBs is comparable to swallowing cherry pits. It’s very unlikely to be life threatening. Typically there is no noticeable effect, or there might be some minor discomfort as it works through the digestive system.
I would be entirely comfortable letting my under 2 year old play with BB-sized steel or glass balls, under loose adult supervision, from a safety perspective (from a that’s-going-to-be-a-mess perspective I can well imagine avoiding giving kids BBs). More realistically, I can imagine letting an older child play with a box of BBs without worrying that a toddler would eat a few and die. Larger marbles are more of a choking hazard, and require closer adult supervision.
But swallowing magnets is much more dangerous.
You'll be spending time in ED having them removed from the nose or ear.
Similarly children often swallow batteries and no one thinks it that’s a good thing but many people do not realize that the child can die from it and it should be treated as an emergency.
You don't need to read a warning label, or even know of the special medical threat of swallowed magnets to think this 'toy' was a bad idea for a small child. Would you give a toddler a box of completely inert culinary grade stainless steel BBs to play with? Of course you wouldn't. Your kid could eat them and shit them out to his hearts content, but he still might chip a tooth trying to bite them and even if he were toothless why would you give a kid even harmless metal balls? They're just going to wind up stuck in his nose or something. Nobody gives inert glass marbles this small to young kids. It's a bad idea all around even discounting the unique threat magnets pose. Which is probably why there are very few confirmed cases of kids actually eating these magnets.