Could it be that they're simply idle sleeping to save power? My first ESPhome device confused me by dropping off and on and it turned out that was the problem, idle sleep was configured by default - the device would wake up, report its status and go back to sleep.