Not really floating, since that implies a certain freedom of movement that atoms in a solid don't have. But otherwise you are correct, since each electron is a tiny magnet (protons and neutrons are also magnets but their magnetic field strength is much smaller so they can safely be ignored). A macroscopic magnet is just a material where the electron magnetic fields add together to an macroscopically appreciable value, rather than cancelling each other out, i.e. the direction of the electron magnetic fields are largely aligned.