4 days late on this but - if an airfcraft is in a sparsely populated area then it is allowed to be below 500' AGL so long as it is 500' away from "person, vessel, vehicle, or structure." This rule is suspended if the aircraft is in the process of taking off or landing (includes "flying the pattern"). I don't know where you live, so it may not apply to you. If you're in a rural-ish area, the person could be taking off from their own land or an airport you're not aware of.