I have a problem that started with version 3.1.6, and still exists in 3.1.8: under enough load, it looks like some connections to the target service are closed too early.
The only log I get from the microgateway looks like:
2020-11-17T21:25:24.699Z [error][api.example.com][1][example-nonprod][non-prod][some/api/path][][][67c658c0-291b-11eb-8fdb-f165ba50582a][microgateway-core][][GET][502][socket hang up][ECONNRESET][]
The target service is behind an AWS load balancer, and the access log for the LB shows that the connection was closed before the service could respond (which is logged as a 460 HTTP code for the service and 0 HTTP code for the LB: connection closed by the client). Apparently, the microgateway then responds with something that look like
{
"message": "socket hang up",
"code": "ECONNRESET"
}and a 502 HTTP code (as logged).
The issue is only visible with enough traffic / load (it's almost never seen on our development environment), which could mean a problem with some resource shared between requests. I realise there's not much information to track the bug, I wonder if there's some log I could activate to provide more. Sticking to 3.1.5 solves the problem in the meantime, but that's not a long-term solution.
Given the limited changes between 3.1.5 and 3.1.6, there's a good chance that the issue is actually in one of the dependencies, such as microgateway-core. However, package.json in 3.1.6 seems to reference a branch of microgateway-core ("microgateway-core": "apigee/microgateway-core#for-3.1.6") instead of a real release. In 3.1.8, it does reference a release ("microgateway-core": "^3.1.9").
I have a problem that started with version
3.1.6, and still exists in3.1.8: under enough load, it looks like some connections to the target service are closed too early.The only log I get from the microgateway looks like:
The target service is behind an AWS load balancer, and the access log for the LB shows that the connection was closed before the service could respond (which is logged as a 460 HTTP code for the service and 0 HTTP code for the LB: connection closed by the client). Apparently, the microgateway then responds with something that look like
{ "message": "socket hang up", "code": "ECONNRESET" }and a 502 HTTP code (as logged).
The issue is only visible with enough traffic / load (it's almost never seen on our development environment), which could mean a problem with some resource shared between requests. I realise there's not much information to track the bug, I wonder if there's some log I could activate to provide more. Sticking to
3.1.5solves the problem in the meantime, but that's not a long-term solution.Given the limited changes between
3.1.5and3.1.6, there's a good chance that the issue is actually in one of the dependencies, such asmicrogateway-core. However,package.jsonin3.1.6seems to reference a branch ofmicrogateway-core("microgateway-core": "apigee/microgateway-core#for-3.1.6") instead of a real release. In3.1.8, it does reference a release ("microgateway-core": "^3.1.9").