Less infectious diseases like colds etc. that spread human-to-human seems likely given the isolation and lower population numbers. Where would the diseases come from?

Of course, if they don't have access to antibiotics, they have other much worse kinds of sickness to deal with, and even their relatively simple lifestyle will be hard to maintain without hard work...

So native Hawaiians.

As I understand it, when corporate start setting up pineapple and sugar plantations in Hawaii they found they could not get cheap labor from the Hawaiian people themselves. And why would they? Between fish and fruit they have all the food they need. A simple structure (on the preferred side of the island) is all you need for shelter.

So they brought in Chinese laborers.