It's even more complicated than that. In addition to the Ainu there were the Jomon people, a non-agricultural but nonetheless settled society who are believed to be the first people in the world to use pottery - which is surprising for a non-agricultural people.
They might in turn have been an offshoot of the first migration of modern humans out of Africa along the south coast of Asia, whose haplotypes [1] occur today at some of the highest frequences in Japan and as far north as Mongolia and as far south as Australia. Some sub-branches spread west all the way to the edge of Europe.
This pattern - early hunter gatherer populations forming the substratum that later mixed with larger migrations of agricultural populations - is not unique to Japan.
It's quite similar to what you find in the paleogenetics and history of most regions of the world, including Europe and South Asia.
We're back to population migration / replacement theories being ascendant rather than the situation 30 years ago when cultural diffusion theories were preferred.
From https://www.nhm.ac.uk/our-science/our-work/origins-evolution...
"When we look at genetic variation in modern British people today, we find that – for those who do not have a recent history of migration – around 10% of their ancestry can be attributed to the ancient European population to which Cheddar Man belonged. This group is referred to as the western European Mesolithic hunter-gatherers. However, this ancestry does not relate specifically to Cheddar Man or the Mesolithic population of Britain. Well after Cheddar Man’s death, two large-scale prehistoric migrations into Britain produced significant population turnovers13. Both of these migrations into Britain represented westward extensions of population movements across Europe10-12. In both cases, these migrating populations intermixed with local people who carried western European Mesolithic hunter-gatherer ancestry, as they moved across Europe. When these populations arrived in Britain they already had some hunter-gatherer ancestry derived from this mixing with local populations. Therefore the majority of western European Mesolithic hunter-gatherers ancestry that we see in modern British people probably originates from populations who lived all over Europe during the Mesolithic, which was carried into Britain by these later migrations."
The two languages are very divergent, and the usual mechanisms for classifying languages are difficult to apply because the dominant Chinese influence came to each separately after they diverged.