News for hackers. Stuff that matters.
Re:Using HAProxy as an API Gateway, Part 1 by Erwin on Tuesday October 02, 2018 @12:54PM (#18123427)
Just how pluggable is HAProxy, or is there anything else out there which allows higher async control over the routing?

My need is some rather customized request routing, currently done by Python (fast enough to do thousands of RPS in just Python with PyPy) but it'd nice to use a standard product.

For every incoming request, I want to decide where it gets routed on a per-request basis to but not necessarily immediately (so some callback / event based / out of process interface is needed) and I want to be notified when the request finishes or fails.

Basically the request for /foo/bar may need to spawn a new foo-bar management process which can take a while to warm up, and appear at some unknown future destination.

I had a look at e.g. Traefik, Kong, Envoy but they didn't seem like quite the right fit.

Using e.g. Proxygen seems like overkill.

Re:Using HAProxy as an API Gateway, Part 1 by sciurus on Tuesday October 02, 2018 @01:12PM (#18123595)
Take a look at OpenResty, which is nginx scriptable with lua.

https://openresty.org/en/

https://github.com/openresty/lua-nginx-module#access_by_lua

Re:Using HAProxy as an API Gateway, Part 1 by graphememes on Tuesday October 02, 2018 @01:16PM (#18123630)
Just a note, Kong is built on OpenResty and has a wide ecosystem of plugins already built in lua with it's own development kit to speed up development time of custom functionality.

Cannot urge how many times it's saved my ass. Not to mention it's very easy to scale!

Re:Using HAProxy as an API Gateway, Part 1 by sb8244 on Tuesday October 02, 2018 @01:33PM (#18123803)
Kong has ran well in production for us, but development can be slow due to the pure LUA scripting. I think some of the problems they have are fixed in an updated version, but there is no update path that doesn't involve re-writing every custom code we have.

We're switching to openresty + LUA proxying to a co-located elixir server that will handle the access requests.

Re:Using HAProxy as an API Gateway, Part 1 by graphememes on Tuesday October 02, 2018 @01:41PM (#18123888)
I'd be very interested in knowing what you've written that you have to re-write _every_ release, especially since you can use the OpenResty API. You'd have to do it for every OpenResty release as well. Doesn't sound right to me!

However, I do say, whatever is easier for you go for it, at the end of the day, it's business value that matters the most! If the refactor increases your business value, do it! If not? Then... don't touch it!

Re:Using HAProxy as an API Gateway, Part 1 by saganus on Tuesday October 02, 2018 @03:38PM (#18124996)
Check out Express Gateway: https://www.express-gateway.io/

Not sure if it would be useful for your case but it's a new-ish API gateway that I've been playing around with, although haven't done anything serious or production-level with it.

Re:Using HAProxy as an API Gateway, Part 1 by caleblloyd on Tuesday October 02, 2018 @10:25PM (#18127219)
You can use the `auth_request` module in NGINX to create a subrequest to an HTTP service that does authentication and/or dynamic routing.

The subrequest can return response headers that are available in `$upstream_http_*` variables, which can br used to dynamically route the original request.

http://nginx.org/en/docs/http/ngx_http_auth_request_module.h...

Re:Using HAProxy as an API Gateway, Part 1 by Erwin on Wednesday October 03, 2018 @07:41AM (#18129277)
Thanks, nginx looks promising, either through this lighter weight integration, or by writing something similar with LUA and OpenResty/Kong.
Re:Using HAProxy as an API Gateway, Part 1 by bedis9 on Wednesday October 03, 2018 @06:41AM (#18129036)
But nginx does not do basic load balancing related features such as health checking, persistence (some apps still need that), DNS service discovery, stats, observability, etc...
Re:Using HAProxy as an API Gateway, Part 1 by imhoguy on Tuesday October 02, 2018 @01:33PM (#18123809)
You can customize HAProxy (1.6+) routing with embedded Lua. Lua support may need to be enabled during compilation if not already provided by OS package. Here are some resources:

https://www.arpalert.org/src/haproxy-lua-api/1.8/index.html https://www.techietown.info/2016/10/haproxy-with-lua-support... https://github.com/zareenc/haproxy-lua-examples

Re:Using HAProxy as an API Gateway, Part 1 by fosk on Tuesday October 02, 2018 @02:53PM (#18124552)
Marco here, CTO at Kong [1]. It seems like what you are trying to implement would be straightforward in a Kong plugin, may I understand what limitations/problems did you run into?

[1] https://konghq.com/

Re:Using HAProxy as an API Gateway, Part 1 by jively on Tuesday October 02, 2018 @11:56PM (#18127534)
Tyk provides pluggable middleware written in Python, it’s very likely you get something like this going because the plugins grant access to the gateway redis cache so you can store data as well as advanced rewrites that can do some pretty complex routing.
Re:Using HAProxy as an API Gateway, Part 1 by bedis9 on Tuesday October 02, 2018 @01:39PM (#18123865)
Well this looks doable with Lua, as an http action probing an api that would let it now if the destination service is available and where. One point about routing, do you mean that server for handling /foo/bar may not exist yet in haproxy's configuration? If so, we could enforce the destination IP at runtime, based on the response provided at the step above.