chosenken

Zerotier really doesn't fail any points, it's just no longer the "hot new thing" or an up and comming startup.

I moved from Zerotier to Tailscale mainly for the fit and finish and Wireguard. Zerotier accomplished everything I wanted for my home network but was never "pretty." Tailscale is "pretty" and easy to use. MagicDNS makes accessing my devices easy as I don't have to set up my own DNS entries.

I moved our entire company off of OpenVPN and AWS VPN to Tailscale. It may not be as cheap anymore, but their networking ACLs and integration with Google/Microsoft for login have made it SO EASY to onboard new people and hardware.

Would it be possible to have a split offering, with both on prem and cloud? In my mind I would prefer to have things like Prometheus, Logs, and Metrics stored on prem mainly due to the volume of logs and metrics we create. Then use Grafana cloud for Grafana Dashboards, Loki logs, and incident management that pull directly from my on prem data stores. I bring this up as it may be cost prohibitive for us to store our metrics in the cloud ( we make so many metrics and logs! ) but I would love to off load hosting the front end. Grafana cloud takes care of managing and maintaining Grafana Dashboard and backend database, Authentication, updates, ect. I'm fine hosting Prometheus and Loki locally, have been for a long time! I just get annoyed having to host Grafana and setting it up, the database up, configuring auth, etc.

Placed on IPFS as well because why not.

http://ipfs.io/ipfs/QmZqRpnRgvBPbP65pdyD2fYhDh9VDEW4EGb5kM41...

I'd checkout HashiCorp's Nomad: https://nomadproject.io/

I think this could be used for both incoming and outgoing requests. From what I gathered reading the docs and API, this doesn't actually forward your request to the API, it just keeps track of rate's and limits, leaving the actual limiting up to you. Its a microservice that keeps track of limits for you, and is highly distributed.

Basically, and I can be totally wrong, what it looks like is you send a rate limit request stating I'm doing 1 hit, I can do 100 hits a minute, am I over limit? Gubernator responds with either a status 0 (no, under limit), how many requests you have left, and the time the rate limit will reset. Or it will return 1 (yes, over limit), the time the limit resets, and you then have to hold your request until the reset time.

So you could use this as a self imposed rate limit for external services. I think the confusion is from the announcement, it sounded like this was a proxy type service that would hold your request until the limit was met. From what I read, it looks like you ask for the rate limit status, then its up to you to actually limit your requests.

Just wanted to say I originally heard about Bvckup 2 when some one posted a link to your little IO Monitoring Tool [1]. I love running it now and then, always felt it was the best little tool to use drive usage! I feel that tool almost deserves its own little dedicated page, else I have to keep searching through the blog to find it.

Thanks for making Bvckup, its one of those tools I never knew I needed till I got it.

[1] https://bvckup2.com/wip/10042018

I have been using Bvckup 2 to backup primarily to my Synology Nas. It correctly identifies that the destination directory is a SAMBA v3.1 mount, and treats it as NTFS.

I actually love using it for my Synology as it has the ability to do Delta Copying. Some of my folders contain VM's and VeraCrypt drives, and it is very fast when it comes to sending delta copies. It still has to read the file on the destination to determine what to write, but that only becomes apparent on large files. I think you may lose some features since it is a mounted network drive(mainly limitations to SAMBA), but I have had no issues with it.

Scanning remote files is fast, if the contents or timestamps haven't changed I've seen it scan over 600K files in a couple of minuets (duplicacy backup, in the process of moving to a new hard drive and need to store it temporarily on the Nas).

Just to note, the $.21/hr broker is on an m5.large (2 cpu, 8 GB mem), which goes for $.096/hr.

We run three nodes right now on m5.xlarge instances. At $.42/hr for the managed kafka, compared to $.192/hr self hosted kafka, I think we'll keep it self hosted for now...

Actually yes. Airflow has cron support baked in, when you run a task you can give it a cron schedule. It then takes care of running your tasks when scheduled.

As for the ssl, I messed up my link. We are in the process of moving the site, so some of the redirects are not working perfect at the moment >.>

We use airflow mainly for ETL work, backfills, and batch processing. We do a lot of work with clickstream type data, be it taking data from analytics.js and loading it into redshift, or taking analytic data from redshift and loading into Google analytics. We have developed many pipes and connectors for airflow that allow us to connect to many data sources, both at the source and sink ends. I mainly work on the DevOps team, running our infrastructure and working on back end systems, so my knowledge of airflow is more high level. I just keep the system up so it can run ;P FYI I work for a startup out of Cincinnati, OH called Astronomer, you can find us at https://www.astronomer.io

Also, we weren't like all down, we just saw lots of time out issues when reading/writing to S3.

Yup, its taking all our stuff down now too. We are working on moving our static site to S3 behind cloudfront, testing it now. Its down.

Our Airflow tasks that use S3, they are all down.

Basically, we are down :)