I looked at the call before and after to see what they had set the buffer to, and they clearly set the buffer to point into what is code. The executable is only 5KB and it's tiny; they had plenty of space in the segment to use a different part of the segment without purposefully blasting their own code.
While it's common, it was still a terrible practice. If whatever was filling in that buffer changed, they could be blasting more code than they intended.
Here's the code from DOS 3.3. I am reasonably sure they didn't intend to overwrite code -- you're probably just seeing a weird artifact where the failure case is leaving a dangling random value that happens to point into valid code.
My guess is that DS isn't being maintained across the failing call to the IOCTL and ends up pointing to the wrong segment.
DOSOutFH DW ? ; fh of DOS destination
DumpMem:
MOV DX,OFFSET DG:BUF+512 ; get offset of bios start
MOV CX,pDOS ; beginning of next guy
SUB CX,DX ; difference is length
JZ DumpDos ; no bios to move
MOV BX,BIOSOutFH ; where to output
MOV AH,Write
INT 21h ; wham
retc ; error
CMP AX,CX ; Did it work?
JNZ WRERR ; No
DumpDos:
MOV DX,pDOS ; beginning of dos
MOV CX,pDOSEnd ; end of dos
SUB CX,DX ; difference is length
retz ; if zero no write
MOV BX,DOSOutFH ; where to output
MOV AH,Write
INT 21h ; wham
retc ; error
CMP AX,CX ; Did it work?
retz ; Yes, carry clearWhere is that published? I was using Github for references on DOS 4 as 3.3 isn't there yet.
(Thanks in advance!)
https://github.com/AR1972/DOS3.3/blob/master/SRC/CMD/SYS/SYS...
Send me a note via email -- I might have some more pointers for you
This is common in code without segmentation protection. CODE and DATA are convention. You can just specify a function, then a small buffer, then another function. .COM files in particular were easier to write with CS and DS pointing to the same region of memory, assuming you could fit both your code and inline buffers in 64kB.
The code explains what they are doing. Even more interesting, they're using their own stack too: