News for hackers. Stuff that matters.
Re:Envoy: 7 months later by tedd4u on Tuesday April 25, 2017 @11:48AM (#14194573)
There was a very interesting talk at Google Next in March that described a way in which the Kubernetes open source team is bringing this sidecar architecture to Kub. In fact I think they are working with the Lyft team. The whole talk is interesting but I've added a time offset to skip to the bit where they talk about sidecar/service mesh.

https://www.youtube.com/watch?v=3quCoi5YHz4#t=27m59s

Also, there is https://istio.io/ but it's a placeholder site at the moment.

Re:Envoy: 7 months later by aledbf on Tuesday April 25, 2017 @03:39PM (#14197232)
You can find the project here https://github.com/istio/manager
Re:Envoy: 7 months later by shawabawa3 on Tuesday April 25, 2017 @01:00PM (#14195577)
Looks interesting except imo the configuration makes it basically unusable.

They use jinja2 templated json... and there are hundreds of lines for their "simple" examples.

Even the "hello world" type example of proxying to google is a mess https://github.com/lyft/envoy/blob/master/configs/google_com...

Re:Envoy: 7 months later by sandGorgon on Wednesday April 26, 2017 @12:18AM (#14200399)
This is a battle that repeats itself ever so often. Same deal with kubernetes config files.

It's the same opinionated positions of puppet vs chef/ansible. Building the tooling in the form of an embedded programming language (lua for nginx, python for ansible) is so much better.

But that is a much more polarising position to take. It's politically untenable. Ruby engineers will say no to python and vice versa... So you start arriving at Json or XML based configurations which are a programmatic neutral ground.

IMHO lua-jit occupies the same political neutrality as XML.

Re:Envoy: 7 months later by henrikschroder on Tuesday April 25, 2017 @03:35PM (#14197198)
I love how history repeats itself.

I remember everyone complaining about Spring and similar having these massive XML configuration files, and then people said "never again!", and moved to very simple JSON formats for their frameworks. And then everyone adds a little bit to the best practices, and over time we get this, again.

Re:Envoy: 7 months later by stickfigure on Tuesday April 25, 2017 @06:30PM (#14198742)
Seriously. How about we just call it EJB 4.0?
Re:Envoy: 7 months later by shouldbworking on Tuesday April 25, 2017 @07:22PM (#14199129)
JSON Server Faces™
Re:Envoy: 7 months later by stock_toaster on Thursday April 27, 2017 @09:14PM (#14216202)
I tried to give it a try, and gave up for now.

1. quite hard to "give it a spin" without using docker (vomit). I eventually found some random github repo that had centos7 compatible build scripts, and it took quite a while to compile it on a test vm since it had to build gcc and a ton of other stuff. Not sure why they can't just supply a static binary "release".

2. The config really is a mess. I managed to get something almost working (only / requests went through the proxy, but nothing else for some reason, even though I was using a prefix config). There aren't really any good example configs I could find either. There is a weird config builder script with json templates in the configs dir, but it just seemed to spew out some preconfigured madness.

I am sure it works great and has more tunables than you can shake a stick at, but for now it seems to be only usable by people who have lots of time to invest in trying it out.

Re:Envoy: 7 months later by mentat on Tuesday April 25, 2017 @11:32PM (#14200226)
That's how they use it, that's not how you have to choose to use it. It is one of the most tedious aspects of the tool but all the different aspects exist for a reason, there just needs to be an easy mode for people starting off.
Re:Envoy: 7 months later by bastawhiz on Tuesday April 25, 2017 @01:40PM (#14196086)
The example you linked isn't so bad. Perhaps a bit verbose for a simple example. The formatting of the json is ugly though, and I think they would do better to not collapse the whitespace inside arrays.
Re:Envoy: 7 months later by stock_toaster on Tuesday April 25, 2017 @03:44PM (#14197284)
Seems like they could have used something a bit nicer to work with too (yaml, libucl, hocon, etc). I am honestly getting tired of editing fiddly json (no comments, last element in list can't have a comma, etc) all over the place.
Re:Envoy: 7 months later by user5994461 on Tuesday April 25, 2017 @05:15PM (#14198096)
You can use hjson for human editable json.

http://hjson.org/try.html

Human editable json, with comments, and doesn't kill you for a lack of trailing comma.

Available libraries in tons of languages.

Re:Envoy: 7 months later by stock_toaster on Thursday April 27, 2017 @10:15PM (#14216437)
That actually looks pretty nice. Thanks for the link!
Re:Envoy: 7 months later by wstrange on Tuesday April 25, 2017 @05:33PM (#14198270)
Given Google's involvement, I am wondering why they did not choose jsonnet?
Re:Envoy: 7 months later by user5994461 on Tuesday April 25, 2017 @07:06PM (#14199034)
Hopefully, it was avoided on purpose because it's a complete nightmare to use.
Re:Envoy: 7 months later by wstrange on Tuesday April 25, 2017 @07:49PM (#14199319)
Honest question: How so?

I have not used it in anger - just played around with it.

Re:Envoy: 7 months later by user5994461 on Wednesday April 26, 2017 @04:16AM (#14201335)
Too much indirection.

You get big files filled with variables, loaded from other big files also full of variables. It's hard to know what you are setting after a while or where it comes from.

After a while playing with configuration systems, I'm starting to think that configuration should just be hardcoded once and for all.

Re:Envoy: 7 months later by Edmond on Wednesday April 26, 2017 @01:37PM (#14205162)
I would rather contact you directly but your profile is empty.

I would like to introduce you to HiveMind described here: http://blog.crudzilla.com/2016/10/managing-configurations-wi...

It addresses the issues that JSSONET does without inventing a new templating language.

I am happy to give a skype demo, email me if interested.

Re:Envoy: 7 months later by josephjacks on Tuesday April 25, 2017 @11:33AM (#14194430)
Envoy is an extremely exciting project. I've been following it closely since OSS inception and am thrilled to see the upcoming deep integration with the Kubernetes project. Such a natural and complimentary fit!

With flexibility as both a side car and host proxy network call fabric for polyglot distributed services, I see Envoy as a kind of modern ESB for the reincarnation of SOA we are seeing in the microservices movement. Congrats to the awesome contributors from Lyft/IBM/Google and others!

Re:Envoy: 7 months later by moondev on Tuesday April 25, 2017 @12:49PM (#14195407)
Interesting that it seems like there is a bunch of overlap with linkerd in the k8s space. With both k8s and linkerd being CNCF projects, I wonder why google is putting so many resources behind envoy rather than linkerd. I guess that's the beauty of the k8s ecosystem though, freedom and choice to use what you see fit. K8s just provides the rock-solid primitives.
Re:Envoy: 7 months later by politician on Tuesday April 25, 2017 @02:58PM (#14196881)
linkerd has a dependency on the JVM, k8s doesn't, and not everyone wants to pull in the JVM if they're not already using it.
Re:Envoy: 7 months later by Spiritus on Tuesday April 25, 2017 @04:10PM (#14197521)
I guess that's one the reasons they created https://github.com/linkerd/linkerd-tcp
Re:Envoy: 7 months later by lobster_johnson on Tuesday April 25, 2017 @09:42PM (#14199856)
One downside to using Linkerd is that unless you have very few apps — or an overabundance of RAM — you can't realistically deploy it as a "sidecar container". The JVM consumes about 100MB of RAM, by their account; it's not lightweight.

The sidecar pattern (used by Envoy and by Google's new project, istio) is desirable in that it compartmentalizes the rules and logic in the pod that uses them, as opposed to sharing all settings across all apps. Zero dependencies (between pods) is a great way to manage complexity. I haven't thought much about it, but there might be latency benefits, too.

Re:Envoy: 7 months later by TheDong on Tuesday April 25, 2017 @11:36AM (#14194456)
Are envoy and linkerd comparable? They look like they fill similar niches, but I'm not familiar enough to know what I don't know.
Re:Envoy: 7 months later by moderation on Tuesday April 25, 2017 @11:47AM (#14194568)
The Lyft team cover the comparison in their docs [1].

1. https://lyft.github.io/envoy/docs/intro/comparison.html#id7

Re:Envoy: 7 months later by sandGorgon on Tuesday April 25, 2017 @11:54AM (#14194663)
This. Linkerd has been a pretty cool piece of tech. In fact, their rewrite linkerd-tcp is in Rust and is blazingly fast. Also, it works on L4 and can do more stuff than Envoy... though I think that advantage is going to be shortlived.

Linkerd guys have been focusing on building deep integration with k8s (as an ingress or sidecar), but this announcement of Envoy trumps it all.

>"We are excited to announce that we are working in partnership with both Google and IBM to bring Envoy to Kubernetes. Fun fact: there are now more people working on Envoy at Google than there are at Lyft! We have a lot of other things planned with Google that we will be able to share more about in the coming months."

That's a bummer for the amazing guys at Buoyant. I had hoped for linkerd to be the next generation of ingress in k8s.

Envoy is in C++, while linkerd-tcp is in Rust. I wonder how much role did the choice of tooling play in Google's decision to adopt.

Re:Envoy: 7 months later by gtirloni on Tuesday April 25, 2017 @12:27PM (#14195093)
Isn't linkerd in Scala and linkerd-tcp a small subset of it?
Re:Envoy: 7 months later by politician on Tuesday April 25, 2017 @12:48PM (#14195400)
That's correct. linkerd/namerd run on the JVM.
Re:Envoy: 7 months later by sandGorgon on Tuesday April 25, 2017 @01:53PM (#14196226)
that's true - but i was hopeful !
Re:Envoy: 7 months later by doublerebel on Tuesday April 25, 2017 @12:34PM (#14195199)
Also see: eBay/fabio, Traefik.io .
Re:Envoy: 7 months later by rdli on Tuesday April 25, 2017 @12:41PM (#14195298)
We've been experimenting with Envoy for a few months and have been really impressed with the technology and the responsiveness of the community. We're actually deploying an API Gateway built on Envoy so you can easily extend your service mesh all the way out to the edge. Planning on open sourcing it soon (if anyone wants to give it a try, my email is in my profile).
Re:Envoy: 7 months later by lunch on Tuesday April 25, 2017 @12:56PM (#14195508)
Any reason you chose to use API Gateway for your edge instead of another Envoy with an ELB in front?
Re:Envoy: 7 months later by rdli on Tuesday April 25, 2017 @01:05PM (#14195663)
Sorry, I wasn't clear. I was referring to using Envoy as an API gateway, not a specific API gateway product. We had to write some wrapper code on top of Envoy that lets you use a REST API to map URLs to services running in your Kube cluster and then dynamically update Envoy config, etc.
Re:Envoy: 7 months later by msoad on Tuesday April 25, 2017 @11:47AM (#14194569)
For Googlers out there: Envoy is basically GFE
Re:Envoy: 7 months later by stellar678 on Wednesday April 26, 2017 @12:32AM (#14200443)
I had no idea what GFE stands for and had to track down [https://medium.com/@jerub/the-production-environment-at-goog...] in order to get a good answer.

Envoy bills itself primarily as a service mesh (seeming closest to GSLB / Global Software Load Balancer), but also as an edge proxy (seeming closest to GFE / Google Front End).

Is there any reason to think that it's more like GFE than it is like GSLB?

Re:Envoy: 7 months later by curiousDog on Tuesday April 25, 2017 @12:31PM (#14195148)
Wut? More like Kubernetes is like Borg.
Re:Envoy: 7 months later by msoad on Tuesday April 25, 2017 @12:41PM (#14195299)
Sorry mixed up. Yes, GFE is more accurate
Re:Envoy: 7 months later by chuhnk on Tuesday April 25, 2017 @12:09PM (#14194863)
More like GFE
Re:Envoy: 7 months later by euyyn on Wednesday April 26, 2017 @01:04PM (#14204853)
Thanks for the translation! :)
Re:Envoy: 7 months later by perfmode on Tuesday April 25, 2017 @10:39PM (#14200054)
I wonder why they chose C++ and not Go.
Re:Envoy: 7 months later by olalonde on Tuesday April 25, 2017 @10:46PM (#14200075)
I presume because the author was more familiar and experienced with C++.
Re:Envoy: 7 months later by searchfaster on Tuesday April 25, 2017 @05:40PM (#14198329)
Any numbers on latency Envoy adds for each request ?
Re:Envoy: 7 months later by searchfaster on Wednesday April 26, 2017 @01:37AM (#14200672)
Just tried it locally, latency is < 1ms..
Re:Envoy: 7 months later by eggie5 on Tuesday April 25, 2017 @06:10PM (#14198590)
according to the talk I just watched, latency/performance was their concern coming out of the gate and was the impetus for writing it in C++...
Re:Envoy: 7 months later by searchfaster on Wednesday April 26, 2017 @01:35AM (#14200659)
Thanks.. just tried it out with my service, beats nginx for simple http proxying pretty easily by a nice margin. Envoy supports way more features than nginx too !
Re:Envoy: 7 months later by rajesht on Tuesday April 25, 2017 @07:31PM (#14199198)
An engineering blog without any diagrams?