I don't think you are giving much credit to ancient roads. The Appian way was built in 312 BC and you can tour it on a regular old bike today. Major ancient cities and trade routes were paved pretty well to accommodate cargo transported on carts. No one likes a broken wheel along the way when they have a buyer at the other end expecting goods. I think a crude bike wouldn't have had trouble like a cart, plus unlike a cart you could hop it up over a bad spot like you do over a curb today or dismount entirely for particularly hairy stretches.