You'd need a sensible way of programming it. Perhaps when 'deep learning' stabilizes a bit there will be commodity neural network chips (there are a few at the moment e.g. http://www.cognimem.com/products/chips-and-modules/CM1K-Chip...).

It's already possible to tweak the process parameters to produce slow-but-efficient chips. I recently saw http://www.ti.com/lsds/ti/microcontrollers_16-bit_32-bit/msp... : 45 microamps/MHz up to 4MHz, runs down to 0.9V. Pair with a suitably low self-discharge battery and it'll run for a decade.