During recombination epoch ~400,000 years after the Big Bang, this light would have been visible. Due to expansion, over time that light has stretched to longer and longer wavelengths, and we currently see it as microwaves.
Note: It's been ~30 years since I was actually studying physics & astronomy; others may be able to offer better explanations or correct me.
> My understanding is that the early galaxies still produced light after recombination.
I'm not sure what you're imagining about recombination, because I've not heard any suggestion of any galaxies existing before it, so they only produced light after recombination.
[0] for some definition of the concept, even though relativity is formulated with the assumption that there isn't any good concept of simultaneousness.
[1] from our point of view. From light's point of view, time isn't defined.
The galaxies we're seeing are in front of that.
[0] 3000 K, https://en.wikipedia.org/wiki/Recombination_(cosmology)