Retrying Requests using Apache HttpClient 4

1. Overview

Apache HttpClient provides support for retrying requests.

In this tutorial, we'll look at how we can configure the request retry behavior for Apache HttpClient 4.

2. Default Retry Behavior

By default, HttpClient retries the failed requests 3 additional times. So the same request can be executed at most 4 times including the initial request execution.

public void executeRetryingThreeTimesImplicitly() throws Exception {
    try (CloseableHttpClient httpClient = HttpClients.custom()
      .addInterceptorLast(new HttpRequestInterceptor() {
          @Override
          public void process(HttpRequest request, HttpContext context) throws HttpException, IOException {
              throw new IOException("Planned");
          }
      }).build()) {
        executeGetRequest(httpClient);
    }
}

Here, we're creating an instance of CloseableHttpClient. To trigger the retry mechanism, we're adding an HttpRequestInterceptor. So when the interceptor throws IOException, HttpClient will retry the request. As a result, the request will be executed 4 times in total.

3. Configure Retry Behavior

HttpClient uses the DefaultHttpRequestRetryHandler class to manage the retry behavior. We can also create an instance of DefaultHttpRequestRetryHandler and configure its properties. Then we must pass this configured instance to our HttpClient:

public void executeRetryingTenTimesExplicitly() throws Exception {
    try (CloseableHttpClient httpClient = HttpClients.custom()
      .addInterceptorLast(new HttpRequestInterceptor() {
          @Override
          public void process(HttpRequest request, HttpContext context) throws HttpException, IOException {
              throw new IOException("Planned");
          }
      })
      .setRetryHandler(new DefaultHttpRequestRetryHandler(10, false))
      .build()) {
        executeGetRequest(httpClient);
    }
}

This example is very similar to the previous one. Here we're additionally creating an instance of DefaultHttpRequestRetryHandler. Notice that we're also setting the retry count as 10. As a result, HttpClient will retry the same request 10 times. If we count the initial request execution, the same request will be executed 11 times.

4. Implement Custom Retry Behavior

We can also provide a custom retry behavior by implementing the HttpRequestRetryHandler interface.

HttpRequestRetryHandler requestRetryHandler = new HttpRequestRetryHandler() {
    @Override
    public boolean retryRequest(IOException exception, int executionCount, HttpContext context) {
        return executionCount < 5;
    }
};

Here we're providing a very basic implementation. It'll retry a request at most 5 times. However, we can also introduce a more involved logic according to our needs.

Then we must register our HttpRequestRetryHandler:

CloseableHttpClient httpClient = HttpClients.custom()
  .setRetryHandler(requestRetryHandler)
  .build();

5. Retry for 400, 404 and 500 Status Codes

By default, HttpClient doesn't retry a request if the status code is one of the client/server error codes - like 400, 404 or 500. This is because there needs to be an IOException for the retry mechanism to kick in - a network failure or an IO error. And the erroneous status codes don't cause an IOException.

Though we can change this behavior.

We must first add a HttpResponseInterceptor implementation. It'll throw an IOException if the status code is a client/server error code:

public void retriesFor500WithResponseInterceptor() throws Exception {
    try (CloseableHttpClient httpClient = HttpClients.custom()
      .addInterceptorLast(new HttpResponseInterceptor() {
          @Override
          public void process(HttpResponse response, HttpContext context) throws HttpException, IOException {
              if (response.getStatusLine().getStatusCode() == 500) {
                  throw new IOException("Retry it");
              }
          }
      })
      .build()) {
        executeRequestForStatus(httpClient, STATUS_500_URL);
    }
}

Here, we're throwing an IOException, if the status code is 500.

After this change, HttpClient gets the IOException and triggers its retry mechanism. But note that if a request is failing with such status codes, the result would be generally the same when retried.

6.  Disable Retry Behavior

Lastly, we can disable request retry mechanism:

CloseableHttpClient httpClient = HttpClients.custom()
  .disableAutomaticRetries()
  .build();

Here, we're calling disableAutomaticRetries() on HttpClientBuilder. As a result, HttpClient will execute each request only once even if the request fails - because of some IO exception.

7. Summary

In this tutorial, we've looked at how we can configure the request retry mechanism for Apache HttpClient 4.

Firstly, we looked at the default retry behavior. Then we investigated how we can configure the retry properties.

We also implemented our custom retry handler and modified the behavior for the erroneous status codes.

Lastly, we looked at disabling automatic retries.

As always the source code for all examples is available on Github.