I have never volunteered to maintain servers since, it was horrible and everything was always my fault (it kinda was, I was a hobbyist at best with no real production Linux experience.)

I do still end up as the dev ops/infra guy at every place I’ve worked but at this point I’m probably one of those stubborn senior guys who wouldn’t like the way the juniors went about it.

Now I'd rather leave hosting to a someone dedicated to it who has internalized the latest state of things for all the relevant bits of software and is constantly keeping this knowledge in their brain. Set and forget self hosting can't work in the current environment we have where things require constant security updates and complex security hardening.

I used to backup to external drives. Now I use bare ones since finding big externals got difficult.

I use (and probably abuse) docker compose. K8s is great but compose is easier.

I use a single makefile. Kinda ugly but it's fine.

Bunch of friends and family use my "services". They usually chip in for hard drives and stuff.

I have a few central points of failure but it keeps things easy. My uptime still beats most big clouds - though I have it easier.

I accidentally took down my server for a few days from a botched hardware install. It's a bit funny because now we realize how critical the home server has become to us.. on the other hand, already got the spouses blessing to build a backup standby server.

Now I have a single dashboard and can upgrade each container with a single click, and everything stays on the happy path.

Maintaining inherited environments is also much more painful than ones you get to design from the ground up. I work with varied environments, and one with ~250 RHEL / CentOS machines has approximately the same level of maintenance burden as another with a dozen or so Ubuntus because the first environment has had configuration management from the beginning and the second is a complete mess that I've slowly tried to reverse-engineer and clean up.

When your change management works, maintaining a dozen servers isn't all that different from maintaining a thousand or more; and the need for change management and automation doesn't really go anywhere even when you don't self-host things.

There's honestly so many posts that make it look so easy, but without everything else that would normally make it a job position in a company :)

All these script kiddie tutorials are terrible at showing how to maintain a server for years.

Docker is great for getting toy projects to work somewhere as a last resort, if the dependencies are strange and you need a convoluted (read: badly thrown together) environment to set up the app.

A well made application should not need docker to run.

Easiest solution is to just host stuff on a local network without access to the wider internet. E.g. running on an old laptop/raspberry pi/server in your basement.

Sure, that means you can no longer access your self-hosted stuff when you're out of the house, but the tradeoff is peace of mind about your data leaking or worse.

Lots of things I'd consider self-hosting are functionally useless if I can't access them from my phone while out and about.

I could put my phone on a VPN, but that's just another layer of complexity to add to the self-hosting process.

1) Have separate bookmarks for accessing services via the LAN or the VPN, and need to remember to turn on and off the VPN when you need to access things out of the house; or 2) Always access services via the TailScale IP, which means you need to be connected to the VPN on your phone even to access them from your LAN, which in turn either means toggling the VPN on and off even when at home or leaving it connected all the time and letting it drain 20% of your battery every day.

It's a great service but I didn't find it to make accessing self-hosted services from out of the home on a phone to be as nice as using them at home, or using a managed service.

I think Tailscale opens up all kinds of new opportunities for self-hosting.

If you don't trust Google drive with your passwords, why would you trust a company's server that manages access to all of your devices?

You have no idea what tailscale is really doing.

please elaborate...

I assumed the GP was talking about a typical home "local network", one behind a NAT - so no incoming traffic, but usually, it allows any outgoing traffic.

One strategy for dealing with accidental misconfigurations is to employ a "network slug"[1]:

"A Network Slug, or "Slug", is a transparent layer 2 firewall running on a device with only two interfaces. ... The purpose of a Slug is to reinforce a security policy or to block uninentional leaks of information."

[1] https://john.kozubik.com/pub/NetworkSlug/tip.html

I would really like to make such devices for home or office use. What would be a good device to use for this? Unfortunately, RaspberryPIs do not come with 2 or 3 NICs. Any recommended alternatives?

https://openwrt.org/toh/views/toh_available_16128

https://www.seeedstudio.com/Rapberry-Pi-CM4-Dual-GbE-Carrier...

... to make firewalls and bridges with rpi cm4 ...

It also does “waterfall” egress packet delaying.

A slug should not need to be dynamic nor should it be complicated in any way ... in fact, it is one of the simpler systems I have ever deployed ...

I have implemented firewalls similar to this in the past. They typically had three network interfaces. Two of them were configured as bridges and then I use ebtables/iptables to filter traffic flowing through. These two interfaces would have no IP address and would not be visible on a traceroute, etc.

The third interface would only be connected to a separate admin network. Or it might not even be plugged in. In the latter case, the admin needing to change anything on the device would have to be physically present and bring a "crossover" ethernet cable and plug their laptop directly into the third NIC of the firewall. From there, they would be able to ssh into the firewall and change config.

Let me give you an example - I have a "port 22 slug" and what it does is block all traffic of all kinds except for TCP22. That's it. It does nothing else and it does it transparently without having an IP address of its own. If I wanted to reconfigure it, I would connect with a serial console.

Make sense ?

Almost like an overglorified but managed hub.

If you like your MAC, you get to keep your MAC.

I'm pretty sure we all used to that and it was mostly fine.

I get that the mainstream computer user has been lost to techno-infantilism. But why should we?

I recognize that this embarrassment of riches is in part my own fault. But this is my answer to your “why”

If it becomes simple again I would gladly self-host.

It doesn't even require actively flipping switches, but can be from not knowing a vulnerable feature was enabled by default. My QNAP got hit with ransomware because of a vulnerability in the cloud access software that I wasn't even using. I've since locked down all non-local traffic.

I'd say the point being: I'm a software engineer who knows better about these sorts of things and still got caught with my pants down. You have to be very judicious with respect to security. You can't just plug and play and say "I'm too busy to worry about that."

Another thing I'll add is the amount of software tools they have on these NAS machines strikes me as 1) very impressive for a company their size and 2) a huge surface area rife for being hacked. When it happened I wasn't surprised at all.

I've since stopped using it because at the end of the day I'd rather pay Dropbox to have peace of mind.

_anyway_

I haven't been hacked.. yet. /me knocks on wood

The precautions I take are basic:

  - Use unique and secure credentials on each service I expose.
  - I only expose ports 80 and 443 to the public. 80 HTTP redirects to HTTPS/443
  - I keep my software updated (docker-compose pull)
  - Nightly backups to cloud storage and local disk
  - I "airgap" my home network from my hosting network. There is no shared hardware between them including firewalss/routers, switches, etc.

Personally, I tend to self host the things whose maintenance I at least find satisfying, and hopefully enjoy. Otherwise I pay someone (through ads or my own money) to do it for me.

I'd like to believe the engineers running Google Photos or iCloud are spending a lot more time on keeping my photos secure and available than I would be willing to put into a server running in my basement.

In the case of a business hiring an MSP to manage something complex like firewalls, Active Directory, server patching, then sure it's reasonable to assume that if they made a mistake, the impact would be equivalent to you making the mistake yourself.

It's possible you need to tell whomever you are reporting to for these post-mortems, they should be outsourcing to reputable service providers in order to free up time and man-hours, not necessarily just to save financially. I suspect that is the real problem.

Big Hint: AWS has had something like 5 times the downtime my biggest clients on-prem datacenters have had this year. So...no, the FAANG engineers aren't doing better than my clients.

Maybe I'm too old (experienced) and cynical, but I always read that as "apps that are going to brick themselves". No thanks.

The problem is, that one can't reasonably wait a few days on every update and look online for breaking changes. Especially with mobile apps that have sometimes a really unreasonable update frequency.

I still have not found a satisfactory solution for me personally.

Drop in replacement while outside LAN are admittedly a little harder and more at risk of mistakes

automation is not a thing? I'm pretty all cloud providers do it...