Assuming you know all of those things, you could go vote as your neighbor. But there are strong incentives not to: if your neighbor votes either before or after you, the double-count will be noticed and audited. Your neighbor will be able to prove who they are, and you'll have walked into an incredibly easy-to-prove criminal charge. Similarly, for in person voting, you run the risk of being identified when you come in to vote the second time (presumably you aren't going to vote just once, since there's no point in the crime if it counts the same as your ordinary vote).
Adding photo IDs would not meaningfully change the security model here, but would give pollworkers pretext to exclude lawful voters ("you don't look enough like your picture").
Edit: and, to be clear: this all makes sense because studies have consistently shown individual voter fraud to be virtually nonexistent in the US.
In other words: in many places, you don't need to present an ID because you've already been identified. But it's a big country, and that's not the case everywhere: lots of states and municipalities do require you to present an additional ID, or at least to present your voter registration (and just your name and address).