The cells themselves also very much are up against physical limits. Last I checked, the theoretical maximum efficiency of a silicon cell in sunlight at the surface of the Earth was something like 29%. The best cells made aren't far from that, and the cells you can actually buy and put into panels en masse are 20-24%. We can talk about multijunction III-V cells, but those are exotic and are unlikely to ever become cheap.
Given the upper bound of efficiency with commodity materials and construction, the only "squishy" places for further cost reductions are improvements in manufacturing efficiency, installation efficiency, and regulatory overhead. On my own home - where I'm installing a 14 kilowatt array - the regulatory overhead is close to 30% of the total cost. That's large, yes, but even if we snapped our fingers and zeroed it, we're still not talking anything close to the gains we've had in the past.
I completely agree that further improvements are going to be "a couple percent here and there" but I don't think that there's a whole lot of total room to cut. We're just not going to see another 10x reduction like we've seen a couple of times since PV became a thing.
We are not close to really any sort of physical limitations. We can continue to use less and less material, and come up with designs that require less and less labor per watt.
And there are entire directions we haven't really explored yet to reduce costs even further. For example, panel lifetime. If we can extend panel lifetime by 5 years, that's a 20%-25% reduction in costs. Yet lifetime has stayed fairly constant and unoptimized up until now. 10 years from now we will have a ton of new lifetime data from existing panels that can help us guide the next generation of panels.