News for hackers. Stuff that matters.
Re:Deploying Mastodon on Digital Ocean by open-source-ux on Thursday November 15, 2018 @06:40AM (#18458494)
At the risk of hijacking this discussion, why is installing server-side web apps so ludicrously complicated? It's madness.

I'd love to see more developers succeed with their self-hosted web apps. But it simply will not happen when the barrier to installation is so high. Too many developers vastly underestimate just how complicated installation is.

Or maybe they prefer the complications because it makes SaaS a much more attractive option to customers when there isn't a simple one-click install option?

I asked this question previously: https://news.ycombinator.com/item?id=18385497

I just find the situation exasperating. Would love to hear why people think this is a non-issue. And let me repeat: a simple, easy-to-install option for server-side software would unlock countless opportunities for developers.

Re:Deploying Mastodon on Digital Ocean by gambler on Thursday November 15, 2018 @10:40AM (#18460067)
>Would love to hear why people think this is a non-issue.

Most developers have never seen anything better. They start working within an artificially complicated ecosystem, get used to it and perpetuate all of the assumptions the ecosystem makes when they create their own products.

And when you point this out, most people just get angry, because for them it suggests they have been doing something wrong for as long as they've done engineering. This is why you always hear things like "buy my website is an application" or "but it has to be complicated to use, because it had complicated functionality".

Simplicity is not easy or intuitive.

I've only started to learn the principles of designing simple systems that do complex things around 10-12 years in my career, after accidentally discovering certain people's talks online (Alan Kay, Rich Hickey) and learning a bit about old-school AI systems. Most people aren't familiar with any of that stuff and don't really care.

One thing I personally cannot comprehend is why web applications are almost universally installed and managed through command line, rather than web interfaces. The only advantage of a command-line interface is pipe chaining, and it's never uses installation/configuration. There is nothing fundamental about command line. It's just a software simulation of a teletype terminal.

Re:Deploying Mastodon on Digital Ocean by tome on Thursday November 15, 2018 @01:29PM (#18461864)
> cannot comprehend is why web applications are almost universally installed and managed through command line, rather than web interfaces. The only advantage of a command-line interface is pipe chaining

Repeatability, scriptability, debugability ... ?

Re:Deploying Mastodon on Digital Ocean by duggan on Thursday November 15, 2018 @11:50AM (#18460830)
The command line has the advantage of presenting portable, terse and editable instructions that can be cobbled together into a recipe which can be abstracted and automated.

It's a very powerful interface.

Re:Deploying Mastodon on Digital Ocean by gambler on Thursday November 15, 2018 @01:18PM (#18461766)
Web interfaces are used by half the population of the planet. Any command you run in a shell can be represented by a URL, an HTML tag or a form. URLs and HTML have editable text representations, and they also have semantics, standard composition rules and generic UI.

Any "recipe" you're getting is likely something you get from a web page to begin with. That can be a tool for composition and automation on it own.

Re:Deploying Mastodon on Digital Ocean by kaishiro on Thursday November 15, 2018 @04:46PM (#18463712)
So very few of those web interfaces used by half the population of the planet actually utilize semantics, standard composition rules, and generic UIs - even though they're available.
Re:Deploying Mastodon on Digital Ocean by craftyguy on Thursday November 15, 2018 @09:47PM (#18465905)
> Any "recipe" you're getting

Not necessarily. Distro package maintainers use and construct recipes that are not web pages, in order to allow folks to install complex software with, in many cases, one click.

Re:Deploying Mastodon on Digital Ocean by yebyen on Thursday November 15, 2018 @07:14AM (#18458620)
Are you talking about these instructions in particular? What exactly is overcomplicated if you don't mind me asking?

This is not some trivial web application, it's a federated social network, and it's all laid out for you, basically up to and including steps for ensuring security and production readiness.

I am exasperated too by this stuff, and I am a professional that works with it every day (but these instructions are especially well laid out.)

I'm counting on Helm to save us. I've been admining a Jenkins server on Helm and Kubernetes and I just haven't seen the "miserable experience" I've heard of other Jenkins admins describing.

I've run several other projects using Helm and I'm convinced it's a pattern of admin-friendliness that is common to stable and well-maintained charts. They are just generally easier to work with, don't feel cobbled together, and you basically never get Configuration Drift.

But it still won't be universally easy until Kubernetes is ubiquitous. It seems easy to me now, because I've already grown accustomed to basic Kubernetes maintenance and it's mostly optional, thanks to the proliferation of managed Kubernetes providers on-the-cheap.

It's still a lot to reason about, and at my place of business the status quo is simply worse than that because of struggling with adoption.

Re:Deploying Mastodon on Digital Ocean by fenwick67 on Thursday November 15, 2018 @09:04AM (#18459277)
> This is not some trivial web application, it's a federated social network

To be fair, its cousin Pleroma is comparatively easy to install

Re:Deploying Mastodon on Digital Ocean by yebyen on Thursday November 15, 2018 @09:20AM (#18459377)
Is it though? I had to dig for a while reading "what is/about pleroma" documents to find the link to the install doc, I think this is it:

https://i2p.rocks/blog/the-magical-world-of-pleroma-setting-...

Is this easier? It looks at least a bit more complicated at a glance to me. I haven't honestly tried either of them, but neither of these are simple to install. Add a config file to nginx, install erlang and use mix to grab the dependencies... these things are not totally foreign to me as a Ruby programmer, but I could see how a newbie would trip over them.

I still don't know which part people are tripping over, I'd like to hear from GP... The Pleroma doc might actually be simpler because it skips over all of the "Hardening your instance" stuff, but I'm only guessing about what people see as the hard part, since none of this looks particularly hard to me.

To be equally fair, it's really never as simple as "docker run someone/image:latest" – that's only the first step, so you can see it running. In Kubernetes terms, that's the equivalent of firing off a "run-once" pod. In other words, if that's all you did then it's not deployed yet.

Re:Deploying Mastodon on Digital Ocean by rogerbraun on Thursday November 15, 2018 @10:02AM (#18459721)
Re:Deploying Mastodon on Digital Ocean by yebyen on Friday November 16, 2018 @10:16AM (#18469249)
(so what's easier about that?)
Re:Deploying Mastodon on Digital Ocean by fenwick67 on Friday November 16, 2018 @07:16PM (#18473026)
It was a few months ago and I don't recall the details but I tried installing both, and Pleroma's intall process was noticeably easier. So it's just my experience.

Mastodon also has this reputation on its own platform.

Re:Deploying Mastodon on Digital Ocean by jeroenhd on Thursday November 15, 2018 @07:18AM (#18458635)
Because most server-side web apps are complicated. The ones that aren't (too some extend) can just get [apt-get|dnf|yum|pacman]ed into your server and you're ready to go. WordPress and Roundcube are one command away from running on your server. If running apt-get, picking a database password and setting your domain name are too much to ask, it's likely that someone like that might not have the expertise to securely run a web-facing server-side application securely.

There's several systems set up to make self-hosting projects easy. It usually involves some behind-the-scenes docker containers with custom scripts and a custom web interface. If you're interested in something like this, try Sandstorm: https://sandstorm.io/ It has a free demo that shows exactly what it can do. It works quite nice, actually.

As you might have seen, there is only a limited amount of software packages available on Sandstorm. The trouble comes with dependencies and overhead. Not every project can talk to Postgres (WordPress mostly relies on MySQL), leading to multiple databases running at once unless you go and manually configure addons and config files to make software talk to each other. Software is designed to run well on a specific base system, be it Ubuntu Server with postgres, CentOS with MySQL or Arch with sqlite.

To make this easier, large software packages are usually packed into Docker containers. Running multiple Docker containers still requires some effort (you need to specify what persistent data needs to go where) but that's nothing compared to manual installation. It's also a requirement one cannot easily get around, as most people/companies have their own preferred storage paths and data storage requirements for important data (think encrypted databases, separate encrypted disk mounts, etc.). This is tiresome and requires reading up on the relevant documentation, but not difficult.

With most of the complexity removed (just ports and paths), you quickly end up running multiple versions of Nginx and Apache at once because every docker container comes with its own web server and its own specific, non-compatible add-ons. This wastes RAM and CPU which could be used to have your services run more smoothly. Most big applications therefore are designed to run best on their own separate hosts to take full advantage of the (virtual) hardware.

The article linked is a good example. It is not a manual about writing config stanzas and downloading packages; it's the bare minimum one needs to handle to configure something like Mastodon.

Re:Deploying Mastodon on Digital Ocean by nullcipher on Thursday November 15, 2018 @10:28AM (#18459970)
I think cloudron provides exactly this. Have you tried it? Mastadon package is also under way.
Re:Deploying Mastodon on Digital Ocean by trystero on Saturday November 17, 2018 @01:34AM (#18474302)
Cloudron.io installation experience is the best I've ever seen. Even easier than Wordpress:

wget https://cloudron.io/cloudron-setup chmod +x ./cloudron-setup ./cloudron-setup --provider [digitalocean,ec2,generic,ovh,...]

https://cloudron.io/get.html And that's it, after that it's only a web UI.

Does anybody know other examples of software which provides command line install personnalized to the provider you use?

Re:Deploying Mastodon on Digital Ocean by synchrone on Friday November 16, 2018 @09:23AM (#18468888)
Shameless plug: https://git.cloudron.io/synchrone/mastodon-app

I am currently working on an end-to-end test suite, as required by cloudron.io platform, to add it to their appstore,

Re:Deploying Mastodon on Digital Ocean by barrystaes on Thursday November 15, 2018 @07:33AM (#18458720)
> a simple, easy-to-install option for server-side software > would unlock countless opportunities for developers.

Docker. Its how any software can be packaged with its dependencies. Best for linux serverside/webapp software.

Install and run a docker with one console command, just provide a network-port and folder mapping.

Or use a webinterface, i like how Unraid does this. If you have a old server/PC to spare, definitely try Unraid.net by just booting it from USB stick.

Re:Deploying Mastodon on Digital Ocean by weberc2 on Thursday November 15, 2018 @10:37AM (#18460038)
Docker only solves the dependency problem for static file dependencies. For things like databases or other services, you still have to stand those up yourself or use another orchestration tool. Beyond that, Docker also puts the onus on the user to solve for networks and volumes and environment variables and secrets management and configuration management.

"Docker" is perhaps part of the solution, but only a very small part.

Re:Deploying Mastodon on Digital Ocean by kamaln7 on Thursday November 15, 2018 @11:59AM (#18460952)
Yep. That’s where docker compose comes in. It takes care of the whole process of linking together different docker containers as well as configuration of each of them (such as env vars).
Re:Deploying Mastodon on Digital Ocean by weberc2 on Thursday November 15, 2018 @04:21PM (#18463471)
Compose is just a declarative syntax for your stack. Ironically it's not composable. In other words, it's not a package definition--you can't ship a compose file that will "just run" anywhere. For example, most web apps require a database, but you don't want to run one database per web app. How do you write a compose file that says, "use the system postgres if available, otherwise spin up a new one". It also totally punts on secret and config management.

Helm is the closest thing to a package manager for Docker, but it targets Kubernetes, not vanilla Docker (and k8s is overkill for a single server, at least considering the time/expertise required to stand up a productionized k8s installation).

Re:Deploying Mastodon on Digital Ocean by kamaln7 on Thursday November 15, 2018 @05:54PM (#18464249)
Ah I see what you mean. Yeah you’re right
Re:Deploying Mastodon on Digital Ocean by orthecreedence on Thursday November 15, 2018 @02:46PM (#18462597)
> Would love to hear why people think this is a non-issue

It is an issue, but like, you have to get over it.

Most of the complications come from dependencies (db, caching, etc). How can I, as a web service creator, make it easier to install and configure a postgres DB? I can't! Software is complicated, and sometimes the requirements for software are complicated.

Sure, I could release a version that uses sqlite but then people will whine that it's too slow and doesn't scale. If I release a version that supports both, now I've introduced an entire layer of complexity that I have to support (and tends to break endlessly in my experience).

Yeah, it sucks. but if you're really that worried about it, like you mentioned, use a SaaS service for the app itself or for its dependencies.

Re:Deploying Mastodon on Digital Ocean by icebraining on Thursday November 15, 2018 @10:00AM (#18459698)
Have you tried Sandstorm? That's exactly what it offers - simple installation and management of server-side apps, sandboxed and using a common authentication and authentication infrastructure.
Re:Deploying Mastodon on Digital Ocean by wolco on Thursday November 15, 2018 @08:17AM (#18458993)
As a php developer I don't share this pain point.
Re:Deploying Mastodon on Digital Ocean by MPSimmons on Thursday November 15, 2018 @08:33AM (#18459068)
It's okay, you have plenty of pain points of your own.
Re:Deploying Mastodon on Digital Ocean by wolco on Thursday November 15, 2018 @06:53PM (#18464686)
I can honestly say developing in php feels great. Php removes the pain points. That's why it has a bad rap because it enables the novice to create and they make novice mistakes.

What languages have a lower pain point level in your opinion?

Re:Deploying Mastodon on Digital Ocean by wild_preference on Thursday November 15, 2018 @11:19AM (#18460475)
Well, yes you do, because the thing that makes this complicated is all the various out of process dependencies like the database.
Re:Deploying Mastodon on Digital Ocean by wolco on Thursday November 15, 2018 @06:36PM (#18464555)
The database is the easiest part in a lamp setup provided you are not using postgres.

If you are using something like digitalocean they have a one click lamp install. It couldn't be easier.

Re:Deploying Mastodon on Digital Ocean by how_to_bake on Thursday November 15, 2018 @07:09AM (#18458592)
You mentioned previously that sandstorm and cloudron are not options. Why not? You’re only talking about hosting on your local machine?
Re:Deploying Mastodon on Digital Ocean by jimmy1 on Thursday November 15, 2018 @10:28AM (#18459978)
Deploying my go web application is as simple as copying a small binary to a server and running ./server
Re:Deploying Mastodon on Digital Ocean by orthecreedence on Thursday November 15, 2018 @02:40PM (#18462536)
So it doesn't have a database (or it has an embedded database)?

With me, the gripes of installing these apps are not just the app itself but the dependencies, which is a necessary evil and impossible to avoid once things get sufficiently complicated.