>Many of these wont accept immigrants from other backgrounds at all, although they might accept low wage agricultural workers.

This is not true. In the case of Japan, so long as an employer there is willing to sponsor your visa, you'll have no problems immigrating there or becoming a permanent resident.