ancieque

I remember installing a metamod plugin to my cs 1.5 Server that added server side fog of war iirc. But i could be mixing things up.

While the way secrets work in Swarm seems weird when compared to Kubernetes, this is usually pretty easily solved by a quick overriding entrypoint in the docker stack file that does essentially this:

    export SOME_VAR=$(cat /run/secrets/some_secret)
    exec /original/entrypoint.sh

Can you explain the second one? I don't get the usecase.

Speaking as a Co-Founder of NeuroForge here, with some exciting news: we're proud to announce a new alpha release for our project, Swarmgate - version 0.7.0. This tool, which started as a simple experiment within our team, has evolved into an essential project aimed at addressing the complex challenges of Docker Swarm management. Whether it's juggling multiple users in a single swarm or optimizing resources across various swarms, Swarmgate 0.7.0 is our latest answer to these issues, offering enhanced features and improvements. You can find it over at:

https://github.com/neuroforgede/swarmgate

What's Swarmgate? Swarmgate is our innovative Docker Socket Proxy designed to provide a tenant-specific view onto a Docker Swarm. It supports all necessary operations for deploying stacks, along with managing volumes, secrets, configs, networks, and more. This allows multiple teams to work collaboratively within the same cluster without interfering with each other, thanks to unique labels that filter requests based on resource ownership. It's about making Docker Swarm environments as efficient and user-friendly as possible.

Under the Hood:

We've built Swarmgate using Node.js and Express, with the dockerode (and docker-modem) library for Docker interaction. This technology stack ensures robust performance and seamless integration with Docker's API, providing a reliable and effective management tool.

What's New in 0.7.0?

    Docker Registry Auth Verification: Enhanced security through Docker Registry authentication checks to prevent unauthorized access to images.

    Security Enhancements: Removal of the :version/swarm endpoint to protect against potential security vulnerabilities by exposing swarm join tokens.

    Simplified Proxying: Introduction of the proxyRequestToDocker function for straightforward proxying of requests without the need for filtering.

    Resolved Log Issues: We've tackled service/tasks log parsing issues to ensure compatibility and ease of use with the Docker CLI.

A Gentle Reminder:

As enthusiastic as we are about Swarmgate 0.7.0, it's important to remember that this is still ALPHA software. It's primarily a defense against accidental disruptions within clusters. While we're diligently enhancing security features, it's advisable to use Swarmgate in environments where there's a high level of trust among users.

What's up next:

    A better tutorial on how to set this up

    More ideas from https://github.com/neuroforgede/swarmgate/issues/1

Cool to see this.

I love DRF for CRUD apis. It just gets the job done and you can Focus on data modelling.

We built our data hub/data Integration solution on top of it. [1] It was a good choice.

By far the most extensible and overridable library I have worked with so far. Even when you need to ressort to hacks, they never seem to break when upgrading a Version.

[1] https://github.com/neuroforgede/nfcompose

If you want to stick on one machine, you can always just use a single node Docker Swarm to get the fully automated zero downtime deploys you want with Docker Compose:

https://github.com/BretFisher/ama/discussions/146

I have implemented this for our tool NF Compose that allows us to build REST APIs without writing a single line of code [0]. I didn't go the route of triggers because we generate database tables automatically and we used to have a crazy versioning scheme that was inspired by data vault and anchor modelling where we stored every change on every attribute as a new record. This allowed for simple point in time queries.

Sounded cool, but in practice it was really slow. The techniques that are usually employed by Data Vault to fix this issue seemed too complex. Over time we moved to an implementation that handles the historization dynamically at runtime by generating historizing SQL queries ourselves [1]. We now use transaction time to determine winners and use an autoincrementing column to determine who wins on ties. A lot of brainpower went into ensuring this design is concurrency safe. On a sidenote: Generating SQL in python sounds dangerous, but we spent a lot of time on making it secure. We even have a linter that checks that everything is escaped properly whenever we are in dev mode [2].

[0] https://github.com/neuroforgede/nfcompose/

[1] https://github.com/neuroforgede/nfcompose/blob/main/skipper/...

[2] https://github.com/neuroforgede/nfcompose/blob/main/skipper/...

Ah and a side note: Superset allows to template SQL queries via jinja. Works very well.

Curious, have you tried speeding things up with e.g. cube.js? We used it in a fully custom project and it was a Performance life saver. It works quite well with Superset actually.

We have looked into metabase ourselves initially as well but found it lacking compared to Superset. What are your thoughts on that?

Note that we build dashboards for customers that end up having some complex SQL directly in Superset.

Docker Swarm engine is part of docker-ce. It just got New features last year as well as some quality of life improvements.

Its not shiny but it works. We use it for most of our selfhosted stuff (which is a lot) and our customers.

I host a swarm fans hangout every 2 months over at devops.fan with Bret Fisher. Also we help each other on discord. Join us :)

Because I didn't like the name, I just renamed it to swarmgate.

CSI support will hopefully make this easier. But: there are quite some options once you Look deeper.

Also the volume plugin spec is so simple that it is possible to maintain your own plugin (even without csi).

What do you need from the managed dbs?

I saw your comment and am wondering what in particular you are struggling with.

I recently fixed one of my biggest pet peeves with docker swarm - the inability to directly exec into a service without first SSHing to the host the task is running on.

https://github.com/neuroforgede/docker-swarm-proxy

Maybe your issue is in this ballpark? Happy to exchange notes on this. If you are looking for a community of Swarm users, check out https://devops.fan (that's a discord hosted by Bret Fisher)

Yeah, had a fun time yesterday when writing a tutorial. Was kinda reliably switching between good and bad every few seconds.

So does this work with Docker Swarm out of the box? I guess this breaks the default networking stack of Swarm, right?

TL;DR: Update your clusters.