I primarily drive and am often nervous driving around pedestrians and cyclists. The reason for this feeling is that I understand, even if subconsciously, how easy it would be for me to harm them with my vehicle. If I imagine myself being the cyclist it's more worrying that I can't control other vehicles that could easily harm me. I constantly remind myself that the road was made for people and drivers are just one class of person -- the most dangerous one to others on the road at that.

The best piece of advice I could give is to slow down and match the bikers' speed if you don't feel safe passing.