The point is, you don't know they legitimately wanted a ladder, certainly not from what the resident knew at the time. All they see is someone who seemed out of place and who may have exhibited the suspicious behavior I described above.

If you've never been in a neighborhood with high crime, or where scam artists have preyed on the naive, great, but don't act judgemental toward those who may have.