Using the Mechanical Turkers to rate fluency would arguably be an even more dubious evaluation benchmark if more rigorous standards and consistent for fluent English existed; people regarded as having good writing and editing skills can find better-paying sources of part-time remote work than AMT. Some of the examples of human editing shown in the blog entry certainly don't look fluent to me...