In the long term, the rift (which is an example of what is, in other contexts, referred to as a "mid-ocean spreading center") will produce much more than a narrow inlet attached to the Red Sea.

But, yes, the initial body of water that will form will not be what anyone would characterize as an ocean.