There is a very good youtube video on this subject.
The Northeast Corridor[1], where the DC-NYC route that was previously mentioned is part of, is owned primarily by Amtrak -- so they get priority on the rails and can run as often (and as fast) as they please. However, they share the rails with other passenger rail lines so both entities cooperate on scheduling and maintenance.
Not to mention that there are sections that are designated as high speed sections, consist of 4 tracks (1 local and 1 express track in each direction), and is electrified. Plus, there are regional transportation agencies (and Amtrak) that operate locomotives capable of 150mph+.
Yes, I know this speed pales in comparison to Europe, but as far as I'm aware, it's still faster than any other rail system in the US.
That's why when Northeast US people rave about Amtrak service (between Boston and DC), and everyone else doesn't, both sides think the other side is crazy due to a myopic view.
What suffers in the Northeast is delivery of freight, so almost all of it needs to go by truck, which contributes to traffic congestion, and drives even more people to mass transit for long distances (which makes it easier to make a convincing case for expanding passenger rail operations). Or to relocate to more densely-populated cities that have rail access.
One interesting by-product of this, is diesel engines are banned in New York City rail tunnels, so the only way to get freight from New Jersey into New York city, is via a 140-mile detour known as the the Selkirk Hurdle[2].
In other parts of the US, the rails are owned by freight companies (BNSF, CSX, etc), so their customers' cargo gets priority, and Amtrak suffers because they don't automatically win the "battle" like they do in the Northeast.
Additionally, in these areas, due to the lack of electrification, Amtrak has to rely on an aging fleet of diesel locomotives[3] that break down often, and when they do, everything gets backed up. Couple this with the fact that the only section of Amtrak's network that's profitable is the Northeast Corridor and there's less incentive to throw good money after bad, and also leads to politicians calling for the defunding or even eradication of Amtrak.
1: https://en.wikipedia.org/wiki/Northeast_Corridor 2: https://en.wikipedia.org/wiki/Selkirk_hurdle 3: https://en.wikipedia.org/wiki/List_of_Amtrak_rolling_stock#L...
It's a shame really! A high speed track between Pittsburgh and Harrisburg would really improve accessibility for the city and be well used I think. I know there's been some talk of it, but no real progress made.
To Boston though I usually fly. It is actually much faster if you leave at the right times to avoid traffic which you'd want to do if driving too. It's about 100% faster in my experience.
I drove to Chicago once and that was about 12 or 13 hours. I've flown a bunch of times and it takes, door to door (depending on when you leave, etc) about 4 hours total. So it's much faster.