It's RAID 5/6, there is a great status page:

https://btrfs.wiki.kernel.org/index.php/Status

This would seem to suggest that ANY raid configuration is unacceptable.

Device replace

--------------

>Device replace and device delete insist on being able to read or reconstruct all data. If any read fails due to an IO error, the delete/replace operation is aborted and the administrator must remove or replace the damaged data before trying again.

Device replace isn't something where "mostly ok" is a good enough status.

This doesn't sound like a bug to me. It sounds like a refusal to fail silently.

If you’re removing a device, it shouldn’t matter if the data is unreadable (that’s probably why it’s being removed, no?)

I interpreted "being able to read or reconstruct all data" as meaning that there must exist a good valid copy of each chunk of data somewhere in the array, but not necessarily on the device that you're trying to remove or replace. That interpretation matches my experience, which is that btrfs can correctly handle replacing a drive that is dead or dying. You should certainly expect errors if eg. both copies of a chunk in a RAID1 are lost/corrupted.

If you have a read error can you expect to recover the rest of the data?

I’d hope so. Just because one bit is missing from a file, doesn’t mean the rest of the files are gone.

Are they testing BTRFS with power interruptions (or simulated equivalents thereof?)