Re:Fixng Nginx 502 Gateway Timeout with proxy_pass During De...
by mfontani on Sunday February 09, 2020 @04:56AM
(#22281312)
The article mentions nginx returning 502 when a backend is down during a deploy. This a problem in the cases where restarting the backend isn't atomic.
If the backend is able to perform a "reload" (vs. a slower stop/start or restart) and/or replace the old instances with the new when they're up -- assuming that's an atomic operation -- nginx will not return a 502.
If one is rapidly iterating and prefers having a somewhat cleaner stop/start, the advice can be helpful even though it masks/papers over a problem (the backend being down).
Re:Fixng Nginx 502 Gateway Timeout with proxy_pass During De...
by simplyinfinity on Sunday February 09, 2020 @04:17AM
(#22281208)
To me the most appropriate way (when not dealing with kubernetes, but rather just vms) is to be able to mark specific server to not receive any new requests, wait for the requests to stop coming in and then deploy.
Re:Fixng Nginx 502 Gateway Timeout with proxy_pass During De...
by notyourday on Sunday February 09, 2020 @09:05AM
(#22282141)
Detecting a failure and minimizing the request wreck that it causes is different from handling a deploy. The second one should cause no wrecks because your deploy management system should be able to tell your proxy layer to drain the target of deploy to allow the deploy to proceed when a target has no traffic
Re:Fixng Nginx 502 Gateway Timeout with proxy_pass During De...
by winrid on Sunday February 09, 2020 @04:12AM
(#22281192)
You have to make a request while the upstream is down. So do the restart, then make a request right away. If your service restarts quickly it'll be hard to reproduce.
Re:Fixng Nginx 502 Gateway Timeout with proxy_pass During De...
by viraptor on Sunday February 09, 2020 @03:26AM
(#22281033)
Is this "fixing"? The proper fix would be to use a seamless deployment by either spawning a new vm with new version, then redirecting the traffic once it's ready. Or by doing the same with a new version of the app server on the same host and replacing the socket when ready.
Otherwise, the requests will potentially wait a long time to be handled while the app starts up.
Re:Fixng Nginx 502 Gateway Timeout with proxy_pass During De...
by winrid on Sunday February 09, 2020 @04:12AM
(#22281189)
In this case it's mostly for experiments and stuff. And spawning a new vm etc is slow for that kind of thing. This takes less than a second (I just merge to master and it deploys).
Re:Fixng Nginx 502 Gateway Timeout with proxy_pass During De...
by winrid on Sunday February 09, 2020 @04:17AM
(#22281206)
also! I worked at a place where this was an issue for years. We eventually moved to Kubernetes, but it would have been nice to know this fix until then for the qa/dev environments where stuff was constantly deploying.
If the backend is able to perform a "reload" (vs. a slower stop/start or restart) and/or replace the old instances with the new when they're up -- assuming that's an atomic operation -- nginx will not return a 502.
If one is rapidly iterating and prefers having a somewhat cleaner stop/start, the advice can be helpful even though it masks/papers over a problem (the backend being down).
IIS web server has this but not nginx :(
Otherwise, the requests will potentially wait a long time to be handled while the app starts up.