You might be thinking about evolutionary learning (or more broadly evolutionary algorithms), which is a machine learning method different from—the more common and more successful—artificial neural networks. Several implementations actually exist, and I think some are even quite successful, but none that I’m aware of have been shown to have AGI capabilities.