The health check services will track the health of available services and send details to the service registry regarding the health of services.
Based on the health of the services, the service discovery process should intelligently route requests to healthy instances and shut down the traffic to unhealthy instances.
Since the services dynamically change their location (address/port) every time a client wants to connect to the service, it should first check for the availability of the services from the service registry. Every connection to the client will also need to have a timeout added to it, beyond which the request has to be served or it has to be retried (if configured) to another instance. This way, we can minimize the cascading failure.