You can look into LFP (aka LiFePo4) cells, which have higher durability than lithium ion. Beware they have different chemistry so charging voltages differ from standard Li-poly or Li-ion (they are also safer, which is nice). I haven't looked at the data, but I suspect supercaps may not be optimized for very low leakage, and generally the energy density is not so good (though if you have access to energy harvesting I guess it may not matter!).