If you want to learn more, something to google would be quantum electrodynamics. There are some nice introductory talks about it, but be warned. In order to really understand this, you'll need to study physics, more specifically quantum field theory.
The electric field and magnetic field are tied together in the manner described by Maxwell's equations. This results in "electromagnetic waves", which are indeed carried by photons in free space.
It is often helpful to look at the fields as being the real "electricity", or at least use the lumped approximation we call "current", and this brings various charge carriers along with it. Electrons are the common charge carrier, but ions can carry charge (e.g. in batteries), and in semiconductors we use "holes", which are the absence of an electron. Not an anti-electron, just the space where it's supposed to be.