Yeah, it is correct. The electromagnetic interaction is mediated by the photon. An electron is just one of the fundamental particles that interacts electromagnetically. We use free electrons in metals for the conduction of electricity, but in the end all the interactions between the electrons, protons, etc are governed by photons.

If you want to learn more, something to google would be quantum electrodynamics. There are some nice introductory talks about it, but be warned. In order to really understand this, you'll need to study physics, more specifically quantum field theory.

The reason there is a potential is because of electromagnetic forces, which are mediated by photons. It’s a bit complicated.