If you're on Android you might need a device sold in the Japan market to get them (seems there are mixed reports on this, would love to hear from someone in the industry - does this stuff use standard GSM features and are these enabled on Android by default? could weird manufacturer skins disable them?).

You also need to be in the affected area to receive the message, although third party apps often let you customize the range or priority of messages to receive.

On my US-sold iPhone running on an MVNO I reliably receive earthquake/flood alerts. But I didn't receive the alert this morning since I live in the other end of the country where it was not broadcast.