others-How to solve `failure to get a peer from the ring-balancer` error when trying to access a service proxied by an api gateway like kong?

1. Problem

➜  ~ curl http://10.2.1.3:8000/testauth/hello/a -H 'Authorization: Basic YXBwOjEyMzQ1Ngo='
{"message":"failure to get a peer from the ring-balancer"}%                                                                                                                                           

➜  ~ curl http://10.2.1.3:8000/testauth/hello/a -H 'Authorization: Basic YXBwOjEyMzQ1Ngo=' -vvv
*   Trying 10.2.1.3...
* TCP_NODELAY set
* Connected to 10.2.1.3 (10.2.1.3) port 8000 (#0)
> GET /testauth/hello/a HTTP/1.1
> Host: 10.2.1.3:8000
> User-Agent: curl/7.64.1
> Accept: */*
> Authorization: Basic YXBwOjEyMzQ1Ngo=
>
< HTTP/1.1 503 Service Temporarily Unavailable
< Date: Fri, 22 Apr 2022 08:34:09 GMT
< Content-Type: application/json; charset=utf-8
< Connection: keep-alive
< Content-Length: 58
< X-Kong-Response-Latency: 0
< Server: kong/2.7.0
<
* Connection #0 to host 10.2.1.3 left intact
{"message":"failure to get a peer from the ring-balancer"}* 
Closing connection 0



2. Solution

After debuging, I found that there are 3 targets in my upstream, one is fine, the others are not healthy, so I got that error.

After removing the unhealthy targets from the upstream, I solved the problem.

Some useful comments on this:

This error, accompanied with a 503 status code, means that Kong could connect to none of the backend services in the upstream pool. As you may be aware of already, Kong load balances (and retries) based on the DNS record, or on a manually configured pool of backend services. See https://getkong.org/docs/latest/loadbalancing/ for a complete reference on Load Balancing with Kong.To determine precisely why Kong could not connect to the upstream services, you should read its error logs, at least.



3. Summary

In this post, I have demonstrated how to solve the failure to get a peer from the ring-balancer error when trying to access a service proxied by an api gateway like kong. The key point is to check the target service’s status. Thanks for your read.