From what I have heard from people involved, the things social service workers will look for is access to food, sanitarian home environment, clothes, and health. Kids who look malnourished, whose teeth is falling out because they never get brushed, school is noticing that the kid is always smelling and hair is dirty, and a general lack of basic clothes. If they then go on a surprise visit and find that there is no food in the house, several week old dishes laying around, then they will add up all the red flags. History of violence and drug addiction naturally adds to the story, and a history of failing to support previous kids.

If I had to guess what the most common flag is it would be a lack of food and clean clothes.