But most modern AI is learnt from data, not programmed, how will we know if our AI in conscious?
We're not there yet and may never be, but since we do not know what consciousness is or what leads to it, we may stumble into it...
Human labor has solved a lot of problems. The obvious shortcut to solving lots of problems is mimicking human labor. Given the opportunity to take a shortcut to solve a problem (especially at a profit) some manager in some business will do it.
And zero people can even describe how a sentient robot could be constructed, even with theoretical future hardware. "We'll have as many flops as some models say a brain has, and programmers are creative" is about as far as anyone can get, but I don't think that qualifies as a plan.
So, it's a fun thing to think about, but like all sci-fi it probably isn't the best basis for societal planning.
In order to perform at or above human level (in a general fashion), the robot will need a firm, comprehensive understanding of the world. Won't this unavoidably involve an understanding of the self as well?
I don't think any chips today have awareness. But it seems to me that you couldn't have general, adaptable intelligence without it.
We can still do the fun stuff of course, like writing and art, and get famous and all that.