It doesn't need to be solipsism - for instance, maybe half of us are conscious.

But if we can't even know that, if we don't even have a test to see whether some human or animal is conscious or not, how can we start trying to figure out what makes them conscious? It seems it's impossible to get to something falsifiable without such a test.

Like, you say a rock isn't conscious. But what about a sponge? An amoeba? How can you answer that if you can only guess answer whether your neighbour is?