I've been in 2 about 20 years ago; both ~5.1. If memory serves there were 2 distict types of shaking. The first was a fairly high frequency vibration that shook the walls and mimic what you see in movies; $#@! flying off shelves, etc.

The second came a few seconds later, and was a slow back and forth horizontal movement of the ground. I remember it being a bit faster frequency than that video shows, maybe about 1Hz, but it seemed like the ground was just moving fairly slowly back and forth a few inches.

I've heard many accounts of these phenomena from others. What I'm having difficulty with is the relative motion of the different pieces of earth. I live on pretty solid ground away from active fault lines. If one of our minor fault lines ever acted up, I could certainly imagine the kind of motion you mention. But if my property "cracked," I'd expect to feel the swaying motion and that each piece of my yard would move in the same direction preventing me seeing the motion. On further consideration that the park in the video is built on a former bay, I supposed the quake could be making the dirt float around on the mushy underside.

The first thing I noticed was an oddness about the crack in the sidewalk. It seems to have cracked perfectly along some crack line that's outside of the sidewalk (not likely - just because dirt cracks underneath doesn't mean the concrete will crack exactly where the dirt does) and there seems to be some rubbery surface where the crack ends in the sidewalk (it stretches as the parts of earth move in different directions.) Not like the pure concrete sidewalks I'm accustomed to, but Tokyo is worlds away and maybe it's not just concrete.

P.S. Why the downvoting on my original post? Is it not a valid question?

That pavement is most likely asphaltic concrete (commonly, though incorrectly, known as "asphalt"), which is a mixture of asphalt and sand/gravel aggregate. It's fairly flexible, at least compared to Portland cement concrete.

I was in the earthquake in Tokyo (a few kilometers from Odaiba where that video was likely taken) and I can say I believe it. Out this far from the epicenter it felt very wobbly. It was like my house was on a block of Jell-o. It felt like a 1-3 htz wave. There was a good three minutes of very strong movement and another severe aftershock a 10-20 minutes afterward.

BTW, The girl having a picnic with her boyfriend when the video starts is shouting こわい (kowai) which means "scary!".