I see the self like the emergence of a car wave on the highway. The wave appears spontaneously when the highway is crowded. It can disappear on its own when the density decreases a little. It exists moment by moment as a result of emerging processes and underlying conditions. It is in a constant state of flux. A great part of it is dependent on the external conditions, not just the cars.
Similarly, external factors are also part of the momentary self. The self is not separated from the external world, but the world plays a part in it. It selects the momentary self from a complex space of possible personalities. Basically, we are the resultant of the stream of experiences and conditions that existed around us all our life. We know that also from artificial neural networks, which are built by distilling their experiences.
Only if you define "I" as something that doesn't match reality, just because people change doesn't mean they aren't still themselves. "I" doesn't refer to some ethereal unchanging thing, it merely refers to the self which is ever changing.