Yes. Generally only requiring a $10 PoE splitter like this: https://www.ebay.com/itm/134500605396
Some N100 class machines draw more power, but many don't, and there are more capable PoE splitters for a few dollars extra.
Use a USB touchscreen.
Edit: Putting a device with permanently attached battery inside of a wall or even on a mount, always plugged in gives me the heebie-jeebies.
Or you look at a mini PC and you really can’t buy one at all for much less than $200 these days. Again, no screen.
But Apple will sell you a refurbished iPad mini for $379 and you’ve got nothing to setup.
I share your concern about running it with the battery all the time, but I think it’s pretty common. I probably wouldn’t put it in my wall but I know of a place of business I frequent often that has one plugged in 24/7 and nothing has happened.
Apple power manages devices that are plugged in all the time, they’ll likely just park the battery at 80%. They are also about as good as you can get as far as hardware quality: Apple sells a bazillion devices and has definitely thought of fire risk.
The other benefit of the iPad is that the accessory ecosystem is vast.
(plus the screen. And ethernet / PoE variants are rare, and not as cheap, so if that's a hard requirement, maybe not for your specific use case)
Setting aside what they're for, Linux handheld gaming devices are kind of a perfect fit for a minor "house computer". Made cheap by commodification. Flexible. Sadly no GPIO in these I think but tack on an RP2350 and we're golden.
Strip or modify the chassis and embed them hidden or with the screen facing out. Kachinng.
Raspberry Pi’s biggest strength is its form factor and low power draw.