devplusops

Try not to get bogged down into syntax issues. If this is homework/you are just starting out, you have two challenges:

* Identifying an algorithm to solve the problem

* Expressing the algorithm you've identified

Work out your algorithm in pseudocode/flowcharts/in your head.

Once you've clearly figured out what program you are trying to write, re-evaulate. Maybe you can slowly change your program piece-by-piece over to your intended strategy, or maybe a rewrite should be easy.

I guess what I'm saying is this: figure out what problem you have that is blocking you. It will be a lot easier to move past it once you know which part(s) of the task you are finding challenging.

The trouble is it depends how you define 'backup'.

E.g. if you are using zfs send/receive to do backups, and there's a bug in zfs send/receive....

Some people advise making sure you backup using different techniques/strategies/software, but in practice that gets quite difficult to manage, and you quickly end up going down a very deep rabbit hold on your quest for independent backup strategies.

Consider this example: I'm aware of at least one instance where a faulty tape drive damaged the tapes used by that drive - in a a way that caused those tapes to damage other drives in the same way. The damage spread like a virus. Unless you have "blue" tapes, and "yellow" tapes, and "blue" drives, and "yellow drives", an issue like that won't be contained.

But you do the combinatorics on all the kinds of issues like that which need to be addressed, and quickly you'll end up spending your whole life backing up data. Which will sort of solve the problem, because you won't have time to create any data which needs to be backed up.

Cool!

The continuous async replication feature you mention would be ideal, especially if it's setup so that I can use consul/etcd/... to automatically elect a "master". If you also added (1) a feature so that writes can be redirected to another instance, and (2) a feature so that I can direct SOME of my reads to a master (reads where I care about consistency), then you'd have a not too difficulty to implement - but also very powerful - distributed version of minio. Maybe something along those lines is what you are planning.

Many instances of minio makes a lot of sense to me, and it seems light weight enough (i.e. resources like memory) that it would work. The only deal-breaker feature which looked like it was missing was the ability to set a disk quota. In multi-tenant environments it's important to have resource limits for reliability reasons. I considered putting each minio instance in it's own ZFS volume to handle disk quotas, but I would have had to write a lot of glue code to make that work from a deployment perspective.

Great work on minio - it's the least painful to use S3 server implementation I've run across so far.

HE.net's free tunnel broker is pretty awesome. They'll even speak BGP to you for free if you have your own allocation, which is no longer that expensive.

Furthermore, simply using IPv6 internally - and you can route it over the internet via VPNs etc - is a huge win.

E.g. you can give each docker container a globally routable address. Globally might be 'within your network', or perhaps you have translators from global IP space to your own internally routable space at the edge of the network.

Don't you have all this with v4? Sure, but not as easily. With v6 you can route subnets and re-route sub-subnets. And you can do this all day long without running out of address space. Doing that in v4 is much harder.

I wonder how much IPv6 plays a role.

Once you go IPv6, you never go back. After dealing with the hell that is limited IPv4 address space, everything just seems so easy.

I tried minio and was really impressed with how easy it is to get started.

But I couldn't figure out how to scale it or make it reliable. Having my entire system depend on a single server not going down is a non-starter for me. Do you consider this use case in-scope?

I've been looking at Skylable instead. While setting it up is a little clunky, it's the only open source package I've found that explicitly supports replication across multiple datacenters - which is even more interesting to me than simply across servers within a data center.

Thank you for both replies to my question :)

Whenever I look seriously at Nix/NixOS, I also find myself looking at Guix.

What makes them different / how can I choose between them? (or should I consider using them both? where is each better?)

SEEKING WORK

Full stack/systems/devops engineer. Most recent projects have been with SaltStack, Docker, MongoDB, and MeteorJS. Experience with a variety of frameworks, programming languages, and operating systems. Experience building robust/scalable/secure systems.

A pleasure to work with :).

Location: Vancouver, BC. Travel or temporary relocation is fine. As a Canadian with a degree, a US work visa is easily arranged.

Especially open to remote and part-time arrangements.

Email: devplusops + at + gmail + dot + com