1. The framework should not define your data model.
2. Many 3rd party apps depend on the existing User and other auth models.
So what do I want ? For authentication there are some common requirements:
- session/cookie management to store a user ID (or other info)
- hooks/middleware/whatever to allow me to check user credentials on each request
- safe hashing/encryption of passwords
- form processing/validation
- integration with OpenID, oAuth, LDAP etc
If a framework, or libraries, provide these then that saves a lot of time and effort, and allows me to create a tailor-made solution.
contrib.auth can be made to do most of these if you still want the ability to play nice with other people's code who expect you to use auth. You'll have to create and manage User objects which it sounds like you're unwilling to do, but you can't have it both ways.
I think having it there is better than not. In most circumstances it saves me a lot of time. If I come across a circumstance that I can't use it then I'm just back to where I would be anyways and will have to write a lot more code and modify 3rd party apps.
When I want to do something more original and specific where I don't need all those apps, Django just gets in the way - and that's when I turn to a more lightweight and flexible framework.
Use the best tool for the job. Sometimes Django is that tool. The problem I have is with companies and individuals who think it's the only tool.
The main reason (in my opinion) that Django has an authentication component is because it has an admin section. That requires authentication. The admin section is something to be jealous of because it's a lot harder to duplicate. Creating a user model isn't that hard. While there have been Rails projects trying to implement an admin system as nice as Django's, they aren't as nice and clean as I'd like. And that's a lot more complex than a simple User model.
And yes, I'm aware that one can do lots of things to extend the Django User model. Examples: while the User model doesn't require an email, you could have the form you build require an email; in Django 1.2, you can have "@" and other email characters in usernames and then just reference the username attribute rather than the email attribute when you want the email; in your controller/view, you could first search for the user by email and, if found, grab the username from that object to pass to the authenticate method. It's more that a User model isn't such an incredibly complex piece of code and I find that different sites often want slightly different things that make it just easier to make one's own.
TL;DR: Be jealous of the admin section, not the authentication system.
contrib.auth
Django is as minimal as you want it to be. People seem to have a hard time not using everything.
Take for example contrib.auth - it defines User, Group and Permission models for you. Frequently this is an issue - what if I don't want a username of 30 characters ? What if I want OpenID authentication instead ?
Sure, you can override this but then you find that a whole lot of 3rd party apps - comments, admin etc - rely on these models, which in turn depend on lower-level functionality such as the ORM or templates. The very fact that you have a framework telling you what your data model should look like just feels "wrong".
Does this mean that you shouldn't use Django ? Absolutely not, there are many projects suited to Django - especially content sites where you are putting together large building blocks with some homemade glue. But I've found it more trouble than worth for smaller projects, or where you have very specific requirements - in these cases (assuming you want to stay in Python) - I'd recommend Flask or Pylons.
Unfortunately Django has become the new Blub framework for Python web development, which means it's used by companies by default, rather than being a useful but limited tool in the toolbox.