Technically speaking, Japanese have kana-based keyboard layout, where a single mora is a single keypress rather than two. However, there is still an extra (second) keystroke for dakuten/handakuten (か -> が), which is not required with romaji input (where it's just "ka" vs "ga"), so it's not a 100% speed increase. So, typing is somewhat more efficient than with romaji, but I think I've heard no one uses that, except for, possibly, professional typists.
In any case, even though mostly everyone prefers the IME on their computers, I haven’t seen any Japanese people who prefer romaji input on their smartphones.
Reading characters is a bit faster, but only because you had to memorize 10,000 of them already. People reading English will similarly chunk many of the words they are familiar with so that they are not so much read as they are recognized.
You can test this by changing your input method to Japanese or Chinese. At least in Japanese, the sound of the character is typed out using english letters (romanji) and various options are presented for autocompletion.
You are right that reading characters is probably faster, though.