I just did this about a month ago, and it couldn't be easier to get from London to Bletchley Park, it's literally right up the road from the Milton Keynes train station, maybe a two minute walk.

Getting to Down House (Darwin's House) from London, on the other hand, while totally worthwhile... what a hassle!