Boric acid is very inert. It tastes like chalk.
The Wikipedia page says there are some some/weak evidence that it could affect reproductive systems, but you would need to ingest it for any effect (and a lot of it apparently).
It's not toxic otherwise. You need very high concentrations: lethal dose is 2.6g/kg. To put that in perspective, table salt is lethal at 3g/kg.
Boric acid still makes nervous but it is completely safe?