DJB's primitives are generally simpler than what came before. They're easy to implement, pretty easy to immunise against timing attacks, and relatively easy to review.

So we're not just trusting DJB here. We're trusting a sizeable community that reviewed his work.