I think self replication and a vast, rich environment are missing ingredients in current RL agents. The human brain doesn't just do intelligent behaviour, it also builds itself up from a single cell. Neural nets don't grow like that, they are lesser, from a point of view. They lack the constraints of self replication - survival and procreation. The richness of the environment and the presence of specific constraints are essential for the development of intelligence. And lots of time to try things out.