And that's the problem. Not any fees NY may introduce.

That’s the point.

Why does New York have to accommodate New Jersey and not the other way around?

Yes, but it’s complicated by the fact that a lot of infrastructure funding is done at the federal level, which should in principle care about people in NY and NJ equally.

We aren’t talking about infrastructure in general though, we are talking about a relatively small area in Manhattan.

At least one of the entry points to that relatively small area is a border crossing between NJ and NY, so I think even if the feds don’t own the infrastructure involved (IDK if they do or not), they at least had to sign off on it.

Just because infrastructure crosses a state boundary, doesn’t mean it requires federal sign off.

Because people from NJ put billions into the NY every year via income tax (no reciprocity with NY, hell they’re even coming after new work from home people) and tons of tourism etc. Its in NY’s best interest to accommodate NJ.

but on the other side people living in NY have to endure it. You can not just make the people living downtown dependent on the woes of the commuters. I can not imagine driving into NY as a large share of commuting mobility to make sense...it's just too big and dense

Excuse my European understanding, but isn't it the state's right to make such policy, and isn't it NY's not to care about it too?

Yes. NY and NJ are fully independent and equal sovereign entities from one another. In general, most of the public road infrastructure is built and maintained by state and local governments (though there is some federal funding provided). Same with public transportation.

The main complication comes from the interaction between the states and the federal government.

This is a great example of how, even when one buys the idea of giving rights to individual states over the federal government, the US state borders are rather unnatural. Someone living in Jersey City and someone living in Manhattan have an incredible amount of shared economic interests. However, the Manhattan resident shares a state government with people living in Buffalo instead. This happens in plenty if places in the US: We have more than one multi-state megalopolis, and along with it, metro areas over a million residents that are split in such a way that they count little in their respective state governments.

In most of Europe, borders have had a whole lot of time to align economic development and political organization. But the US, unlike its corporations, is against reorgs. The fact that we even get to discuss state rights for something that could be a municipal matter is, in itself, a problem.

It's unevenly distributed. I live in a suburb of Philadelphia and I can generally get around by bike, with a relatively frequent train into the city. I use my car something like once a week.

Just north of me is a car sewer though, so your point stands.