bdimcheff

Our backend is 100% go, and we're happy to answer any questions about tech or insurance.

Openly, Inc | Backend Engineer (mostly go) | Ann Arbor, Boston | Remote(US) or Onsite, Full Time | Salary, Benefits, Equity | https://openlyinsured.com

We’re a currently 7-person insure-tech startup with 3 engineers: me in Ann Arbor, the CTO (one of the cofounders) in Boston, and someone who works remotely from socal. We’re building an insurance company that uses technology from this century to make insurance agents’ lives easier. We're obviously a small team, so you'd be able to have a large impact quickly.

We intend on staying a remote-first team, but we're also trying to start a small Ann Arbor office if we can find the right person to join me here. Having a physical presence in a2 will allow us to hire interns/juniors and mentor them and such, which is more difficult to do remotely.

More details at https://www.welovegolang.com/jobs/software-engineer-57359861.... I know it says Ann Arbor only, but you can ignore that.

RBAC itself is enabled by default on new gke clusters afaik, and has been enabled on ours for months. We're a small team so we haven't tried to do anything particularly complex with RBAC inside the cluster, but I can't say I've run into anything particularly troublesome.

We use GKE at Olark for the last year and have been very happy with it.

Yeah I think that there is still some potential in the terminationGracePeriod strategy, but we found this other way that worked reliably and stopped exploring that path. If I can repro the issue I'll let you know.

One extra thing I remember that was sort of problematic was that when a pod was Terminating it'd get removed from the Endpoints, so any tooling that was using the API info to keep an eye on connections was basically unusable at that point.

One thing I didn't put in here that's also turned out to be useful: We can prerelease things relatively easily this way too. Each deployment has a git sha, and we can have a canary/beta/dogfood version that points at an entirely different sha.

In our case it's not the websockets that are the problem, it's the XMPP connection that each websocket connection creates. Logging in thousands of users takes several minutes. While a user reconnects, any conversations that the users are having with their website visitors are disrupted.

Yeah I think if restartPolicy were changeable at runtime, we could simply have the pods exit once their connections are drained enough. If we were to exit using the current strategy, they'd just be restarted by kube.

yeah I don't know why terminationGracePeriodSeconds hacks didn't work. It could have been a different, unrelated factor that we didn't discover. It certainly could have been service-loadbalancer/haproxy's fault instead of the termination grace period itself. I'm certainly happy to be proven wrong there.

We force reconnects eventually, there just aren't that many people affected at that point. There's a very long tail of people keeping their browsers open for days, but it's only a handful of people.

We actually have this functionality as well: we can send a signal to the process which will cause it to display a message to the user asking them to reload to upgrade. There is a similar feature in slack and riot that I've seen as well.

Making this handoff automatic is definitely possible as well, though we do want people to reload occasionally to get new client-side code.

The way this Rainbow Deploy works means that if I only deploy once a month, I only have a single "color" running for that whole month, plus couple days of overlap where there are 2. If I have blue/green, I have 2 colors running all month. If I have more fixed colors, even more. The sha thing is just a convenient way of creating "colors" dynamically when we need to do a new deploy, without having to use a meaningless representation like "blue", "green", "taupe", "chartreuse", etc.

Yes, as lotyrin pointed out below, the backend is already effectively an XMPP <-> Websockets bridge. Every time a user logs in, one way or the other we need to establish a connection between the websockets backend and the XMPP backend. The backend does do other things besides simply proxy, and we certainly could separate out the part that needs to keep the state. This deployment strategy is effectively a way to avoid having to do that work, at least for now.

This is our experience as well. On one service with 3 backends, we lost all connections to one at about 0530UTC, then the second at about 1010. The third backend has been able to handle all of our traffic so far, but we're also seeing intermittent connection resets or failed connections.

Other developer here. I've been running nixos as my main OS since about November after running Funtoo for a year or so before that. My tolerance for hacking on my distro is maybe slightly above average. ;)

On the cool side: Nix is the first linux distro I've contributed to, because it's the standard "hack, fork, pull request" process that most github-hosted open source projects use these days.

The biggest thing is that you absolutely have to drink the nix koolaid if you're going to run nixos. You can't really do the normal ./configure && make && make install, because paths to libraries are all non-standard. But once it all really clicks, it's pretty great.

As far as desktop goes, the rest of this thread rings true for me too: most of the important things are there, like desktop environments, window managers, browsers, etc. There are occasionally niche/older packages that are not available. If you're willing to learn a little bit of nix language, it's relatively straightforward to add most packages.

We do this at Olark to a large extent, but I don't think anybody has written about it in any detail yet. We have a peer-driven project proposal process that allows anybody to propose a project, and then "vote with your feet" to get it done. It's still pretty new, but I'd guess we'll write about it once we have a little more experience with it.

I'm only 5'9", so I don't fully understand the geometry of being tall on an airplane, so forgive me if this is a stupid question: Can't you [tall people] put your legs down under the seat in front of you and avoid having your knees directly in the area where the seat reclines? I do this when I want to stretch out more, but does this not work for tall people?

I have a lenovo x230 that works great for me... It's not as high-res as an X1, but it is super easy to swap in 16G of ram and an SSD. I think the X1's ram is soldered. I also have an extra battery that snaps onto the bottom which will give me ~10-12hrs of total battery life when I'm going about my usual business.

I've been thinking a lot about copyright terms lately, and this is what I have come up with: You get 5 years for "free", just like we have now. The next 5 years cost you $10k. The next costs you $100k. Add a zero every 5 years. It's going to be very difficult to keep a copyright for more than 20 or 25 years, but if it's really worth that much to you, then by all means...

T-Mobile has discounted post-paid plans again if you bring your own device: http://www.t-mobile.com/shop/Packages/ValuePackages.aspx

I'm on the $30/mo "walmart" prepaid plan, but I'm considering switching back to post paid because you can't use any data roaming at all on T-Mobile prepaid.

You can hide the widget by default and then show it with javascript when the user clicks a link/image/etc.

Olark - YC S09 - Ann Arbor, MI (http://olark.com/jobs) is looking for a DevOps engineer.

We're growing fast and we need somebody to help our infrastructure keep up. We're profitable and you'd be #12 on the team.

You'd be doing infrastructure automation with puppet, helping us scale our messaging infrastructure so we can grow, and building tools to help us monitor our servers and our business.

For the curious, we use: Python, Ruby/Rails, Thrift, ejabberd, rabbitmq/amqp, mysql, mongodb, couchdb, redis, node, and more!

Check out the link above for more info on the position and to apply.