I actually had complex joins in mind.
A good ORM (in the sense that most ORMs aren't good by this definition) should retrieve the child rows if and only if they are required for the final result. And it should request exactly the columns that are needed by the result. And, as already pointed out in my previous comment, it should hand over the aggregating to the database - that's one of the main things databases are good for.
Gavin King (the Hibernate guy) argued that all columns should be fetched by default because (A) the marginal cost of doing so is very low once the db has taken the disk hit for any data in the row and (B) an in-memory object cache is easy/efficient if entire objects (rows) are cached at once. I might be paraphrasing a bit much.
One could argue that, when some databases can return a subset of column data from indexes (vs eating the disk hit for the row data) that limiting the columns fetched could be much more efficient.
There are probably very few areas of any app I've worked on where I would have gotten significant performance gains by limiting results to a subset of columns.