Browsers were always non-validating XML processors. Maybe you're thinking about the Well-formedness constraints? These constraints check if an XML document follows the XML metasyntax and any XML processor is required to fail [1] on errors in these contraints. And browsers do that.
[1] https://www.w3.org/TR/2006/REC-xml11-20060816/#sec-conforman...
In practice most well-formedness violations were nesting errors like missing start or end tags, common when generating XML via text concatenation or templates, which in the dark times a lot of people do, even when they shouldn't . [2] That's why the HTML5 parsing algorithm is so complex and almost unimplementable. It's the ambition for error correction: It tries to generate a DOM out of any well-formed and unwell-formed documents.
I believe browsers would be quite within their rights to do DTD validation and consequently reject such a document as this, but they don’t actually do so. (Even if the DTD is specified correctly for XHTML 1.0 Strict, nothing changes.)
Thanks for correcting me. I have updated my mental knowledge repository.