Self hosting is hard because ops is hard [1]. People aren't yet in the mentality that you should provide an official modular deployment every time you provide a server binary. Everything-as-a-container wouldn't be the worst way to take the desktop [2]. Isn't that kind of how Apps work on OS X?
[1]: The ironic part is that most people don't keep anything long enough for a hard drive or any other component to fail anymore. So the argument about the cloud abstracting away the physical hardware maintenance for every day consumers like yourself or I is ... dubious.
[2]: Yes.. I just spent the weekend learning nix-build. Goodness, it's super easy to container all the things! :) I'm feeling like a zealot when I start using nix to do what I could have done with a zip file. But there's something magic to having a zip file of assets with its own shell.
Hypothetically in a future where self hosting is non hostile, will you see people self-hosting startups and the like? Yeah, maybe feasible. I think at that scale you start to care about things like uptime and maintenance. But I think the biggest winner to self hosting is the photographer who saves $5/mo on the blog that people rarely read or the kid who doesn't have to pay $10 to play Minecraft with their friends, or the family friend with 2TB of family data that doesn't want to pay $20/mo for Dropbox or the like when they already have the hard drive to store it. Yes it needs to be simpler for these people! There's a whole economy in making it difficult for these!
Huh, is that not a thing anymore for startups today? In my experience, starting in 2003, across two companies, was that if you need any service fast and on the cheap, you're forced to host it yourself. Run your own mail server, web server, DNS, server housing, all the web apps, etc. and of course spend most of the day developing your actual product. Have the hosted options actually gotten so cheap and reliable these days? Is it a mindset thing?
My gripe with the cloud or SaaS solutions I've used over the years is always that management of them and backup is difficult tending towards impossible and without those you can't rely on these services, it's just voluntary vendor lock-in. When self-hosting, your actually (forced to) learn how they work and be able to fix them. Self-hosting, to me, is the simpler, more reliable option, if you depend on a service for your business and it enables you to get it fixed yourself, when it is not working. It doesn't prevent you to out source the work, if you have the money, but worst case, you still have direct access to your property.
With the hosted options, in my experience, you get all these fancy promises on availability and it being the latest, smartest tech and then the service is down for half a day, your data got restored from days old backups (if at all) and there is nothing you can do except tell your customers that "were sorry and working on it" while you wait for it to come back. :-(
It's not unheard of for the same thing to happen with in-house systems.
The difference is that you may have a wider range of options to avoid and respond to any outage if you run things yourself. That seems to be a rather theoretical advantage though. Every time there's a new ransomware attack, it's always those in-house deployments that are hit hardest and take the longest to recover.
I think under optimal conditions self-hosting is superior. But conditions are rarely optimal. As soon as you have to convince non-technical management to invest in non-productive necessities or in contingency planning you're already in a sub-optimal position.
So, I think, the ideal situation would be to have a combination of big and small companies offering storage and compute as a commodity. You'd pay a fee to keep your stuff hosted somewhere; any time you see a better offer, you can migrate to a different provider without much hassle, and with near-zero downtime. Cloud services would work by shipping their code to your data, not the other way around[0]. And if you were so inclined, you could just build your own infra, or even buy a turn-key "self-hosting in a box" kit.
Pieces of that vision are already here. Compute providers are plenty. You can order "self-hosting in a box" kits. Internet architecture makes everyone's computer equal (at least in theory, ISPs mess it up with NAT, and their T&Cs). The only thing missing is the part where you own your data, and SaaS vendors serve you - the bit that makes SaaS truly be Software as a Service, instead of Serfdom as a Service.
--
[0] - Preferably with homomorphic encryption preventing SaaS vendors from putting their hands in the cookie jar, if we can get that to work without creating another blockchain-level environmental disaster.
Regardless of what you use for hosting your own stuff, if you want to use a third-party SaaS as a user, they own the data. Want to make a document on Google Docs? That document lives on Google's servers, it's forever tied to their service and mined by them. There's no artifact you can hold on to, other than your user account.
What we need is a system where the data for that Google Docs document lives in a place you control - be it your own hardware, or some hosting you rent somewhere. It's the SaaS that should come to the data, and operate on it there. That way, if you lose your Google Docs account, or decide to edit the document with something else, you actually have that document, in its canonical form. Same for all other SaaS.
And there is your niche right there. Individual companies or persons do not need to make 'dents in markets' all by themself. A host of people doing the same might. But for the individual - especially when having sustainable income objectives, not hockeystick growth - there's a good place in the market I think.
I guess none of members of your mentioned groups will ever grow to care about a scale. And thats a good thing because they are distributed. And i think it creates a better internet, which is kind of wide network instead of shallow graph of couple mega nodes.
The real problem is who owns your data. Because if you use Word or Photoshop, your files are locked inside Word and Photoshop. And this is still true if you use FOSS alternatives, because there's only very limited support for metadata-aware sharing between applications of all kinds.
It would be super-useful to have (for example...) seamless links between text editors, web design applications, web hosting systems, and even video editors and ebook publishing tools. But that's not where we are now. There's some limited interchange, but most cross-domain transfers are difficult and fragile, and some are impossible.
Cloud is just the online version of the same model. When you have proprietary control of user data through proprietary file formats which actively frustrate open sharing of data between applications, it doesn't matter of the data is stored locally or in the cloud. It also doesn't matter if you're using a mobile or desktop UI.
The FOSS people have always been looking through the wrong end of the telescope. The real revolution would be open data which is wholly and exclusively owned by users (or user groups for collaboration) and loaned to proprietary software for specific limited tasks.
Which is the opposite of things work now. Applications and products own your data and they let you access it - but only if you ask them nicely. And - increasingly - if you pay annually for the privilege.
So containerisation or self-hosting or whatever is a non-solution unless it also gives data back to users.
Which is also why the fragments idea won't work. There's limited use in trying to automate or manage or otherwise AI-ify access to data that you don't truly own anyway.
In fact a new kind of shared Internet would be a very useful thing. But it would need a ground-up redesign of everything, including browsers, mobile apps, desktop applications, operating systems, search, and the financial and legal frameworks surrounding them.
I'd love to see that happen. But right now in 2021 it just doesn't seem likely.
How would that work in practice? By its very nature open data would also be accessible to proprietary programs although the reverse need not be the case.
Seems like we will need open data and code. Either one open will not do.
That is a good thing. If program X works best on my data I want to use it. There are a few examples where people do mix programs from different companies. Musicians use MIDI to connect their favorite keyboard to a synthesizer from a different company all the time - sure it is tied to hardware, but it need not be and is a perfect example of what should be possible for any user data: mix and match.
> although the reverse need not be the case.
It doesn't have to be, but if users demand it, it will be.
> Seems like we will need open data and code. Either one open will not do.
Open data means we can create the code. Closed data is a lot harder to deal with than closed code.
When your data is complex, the processing done to it will be complex, especially if you need to guarantee invariants (eg. referential integrity or database constraints).
Edit: I envision it a bit like streams of data that you can subscribe / push to. Think RSS mixed with a pub/sub type of model. You would subscribe to the hackernews datastream, and submitting articles and comments are done using push. The push message would have some predefined metadata fields that are obligatory (article url, title, summary or comment text).
Can you imagine if Android licensed iMessage instead of building Hangouts? Yes, we’d all be texting on the same protocol, and yes we’d have a choice of clients, but at what cost?
Yes this is kinda how apps work on macOS, but not all apps yet. The new sandboxed container model isn't mandatory yet. Some apps are just a folder of files with no kind of containerisation at all (other than the .app itself being a folder rather than a file). More modern apps store all their data in a containerised filesystem though.
It leads to some cool things you can do like easily capturing the icon of an app or even changing it without having to change the app itself.
But on the self-hosting side, docker is making big strides there. Almost everyone deploying something like Home Assistant for example will do it through a tree of managed docker containers, in many cases without even knowing it :) It comes with a supervisor (which is also dockerised) which manages that part very well.
I think most consumers I know keep their computers for long enough by the way.. Phones come and go, but desktops and even laptops tend to stick around until they do break and I can't fix them anymore or just heave a sigh and go like "NO this time you really have to get a new one, Windows Vista hasn't been supported for years"
Docker is only part of the puzzle. Even being proficient with Docker (and running services in a traditional way), there was always a big barrier to self-hosting for me in networking. If you lose all of your cloud-equivalent functionality once you leave your home network, it isn't a realistic alternative. I'm not a network whiz and the builtin VPN functionality of my NAS is sadly not up to par either.
In comes Tailscale[0], which made it dirt easy to self-host my stuff and have it available anywhere (given there is a Tailscale client for it which has been the case for all my devices). Since I started using it, I've completely migrated my contacts, calendar, zettelkasten (Trilium) to self-hosting and started some home automation projects. For someone who tried and was stuck at the networking part in the past, it truly is a game changer (and I'd like to think I'm not the only one that is held back by that).
There's also ZeroTier and Nebula. Mesh VPN is definitely here to stay!
Previously discussed here: https://news.ycombinator.com/item?id=24027663
Where do you see this?
> More and more people seem to be doing this.
Which people?
My money's on your anecdata being 100% from a techie bubble.
In reddit, colleagues around me etc. Hacker News posts/initiatives. 2-3 years ago everyone was all "Hey look at my new Office 365 setup". Now the cool new thing is more self hosting, often driven by a desire for more privacy. There's also new businesses around this usecase. Look at https://www.beeper.com/ for example. It's a hosted matrix service but all the bridges containing your private data are self-hosted.
> My money's on your anecdata being 100% from a techie bubble.
Absolutely. But this is where things start before they get mainstream. Things pick up traction here. Then they mature and commoditise and make their way to the mainstream.
1. Where is the money in that aka is there more money in that than in services?
2. Will it be braindead easy to use?
My money is on no and no so I don't see how the chasm crossing will happen.
I agree with your post in general but not this. I see a lot more interest in self-hosting stuff lately, precisely because of the concern you mention that online services do ads and tracking. More and more people seem to be doing this.
And I personally really prefer native apps over web apps or electron stuff.