ISPs don't really want to charge both parties so much as they want to tax Netflix for competing with VOD services that ISPs (who are generally also digital TV providers) want to provide.

Its not really about getting money from Netflix, its about getting rid of Netflix.

Two parties benefit when you order a movie on Netflix: you, and Netflix. The company that builds the infrastructure should be able to charge both. It's just how public transit infrastructure is often funded by a combination of user fees and increment taxes on the businesses served by the new road or train line.

Indeed, it's a fairly basic principle of the economics of building infrastructure. A free market underinvests in things like train lines because the builders can only capture one aspect of the positive value created by the infrastructure (the value to the user). However, that leaves a large amount of value to the businesses that users use the infrastructure to get to. If builders can't capture that aspect of value, they'll under invest in such infrastructure. There is a whole body of work on this subject.

Netflix pays a company to provide Internet infrastructure to them. (Level 3, I think) If your ISP isn't Level 3, they should not have the economic right to charge Netflix, because your ISP doesn't service them. Customers pay their ISP to deliver things customers want, no more, no less.

Even if Netflix pays someone to get its data to Comcast's or AT&T's network, they benefit enormously from Comcast or AT&T getting that data to the consumer. In theory, both intermediate ISP's should have the right to charge both endpoints.

And in fact, that's how it has traditionally worked on the internet. Netflix would pay it's ISP, and the consumer would pay their ISP, and the ISP's would pay each other to account for any asymmetry in the data transfer between them (peering). These payments would ultimately be reflected in the charges billed to both endpoints, so the ultimate effect was that each intermediate ISP effective billed each endpoint of the connection.

This whole debacle arose because Netflix's ISP (Cogent) wasn't willing to adhere to the traditional peering arrangements. So in the end, Netflix ended up paying to connect directly to Comcast's network.