In practice, light gathering correlates to F-stop quite well on most normal modern lenses, transmission losses are very low. The main exceptions are deliberately apodized lenses, which have considerably transmission losses wide open.

For the rest - yeah, f# is just one of important specifications.