Apache HttpClient hangs

Recently we were trying to hit a website in multiple threads and get the response to improve the speed. But unfortunately the HttpClient hangs when we try to hit it from multiple threads. Following is my code which does not work.

import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;

import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;

public class HttpClientTest {
	public static class CallableRequest implements Callable<CloseableHttpResponse> {
		CloseableHttpClient client;
		
		public CallableRequest(CloseableHttpClient client) {
			this.client = client;
		}
		
		@Override
		public CloseableHttpResponse call() throws Exception {
			HttpGet request = new HttpGet("https://www.yahoo.com");
			CloseableHttpResponse response = client.execute(request);
			return response;
		}
		
	}
	public static void main(String[] args) throws IOException, InterruptedException, ExecutionException {
		CloseableHttpClient client = HttpClients.createDefault();
		ExecutorService executorSerivce = Executors.newFixedThreadPool(10);
		List<Future<CloseableHttpResponse>> respList = new ArrayList<>();
		
		for(int i=0; i<10; i++) {
			CallableRequest req = new CallableRequest(client);
			Future<CloseableHttpResponse> future = executorSerivce.submit(req);
			respList.add(future);
		}

		for(Future<CloseableHttpResponse> respFuture: respList) {
			CloseableHttpResponse resp = respFuture.get();
			System.out.println("Status code:" + resp.getStatusLine().getStatusCode());
			resp.close();
		}
		executorSerivce.shutdown();
	}

}

The HttpClient looks like to have an internal queue and process a limited number of open requests. It can process new requests only after response of earlier requests is closed.

I am closing the response after reading it, so it should free up the open requests and it should work. Right? But it does not work.  

It is not working because I am submitting 10 requests and waiting for response in sequence. But the sequence of submitting the request and execution of them by HttpClient can be different. If HttpClient can process only 4 concurrent requests and your first request happen to be 5th request in HttpClient then you will be waiting for your first request to complete and it will never complete because some other 4 requests need to be closed for your first request to be processed. The 4 request which are completed are later in your list. That is why this program hanged.

The Solution

The solution is to wait for results in sequence of completion and not in sequence of submission. For doing that you need to use ExecutorCompletionService for submitting your request. It has a method take() which returns you next completed task. You can process the results of you requests in sequence in which they are getting completed. This way you HttpClient will not get locked up. Following is the code:

import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ExecutorCompletionService;
import java.util.concurrent.CompletionService;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;

import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.util.EntityUtils;

public class HttpClientTest2 {
	public static class CallableRequest implements Callable<CloseableHttpResponse> {
		CloseableHttpClient client;
		
		public CallableRequest(CloseableHttpClient client) {
			this.client = client;
		}
		
		@Override
		public CloseableHttpResponse call() throws Exception {
			HttpGet request = new HttpGet("https://www.yahoo.com");
			CloseableHttpResponse response = client.execute(request);
			return response;
		}
		
	}
	public static void main(String[] args) throws IOException, InterruptedException, ExecutionException {
		CloseableHttpClient client = HttpClients.createDefault();
		ExecutorService executorSerivce = Executors.newFixedThreadPool(10);
		CompletionService<CloseableHttpResponse> completionService = new ExecutorCompletionService<>(executorSerivce);
		List<Future<CloseableHttpResponse>> respList = new ArrayList<>();
		
		for(int i=0; i<10; i++) {
			CallableRequest req = new CallableRequest(client);
			Future<CloseableHttpResponse> future = completionService.submit(req);
			respList.add(future);
		}

		for(int i=0; i<10; i++) {
			CloseableHttpResponse resp = completionService.take().get();
			System.out.println("Status code:" + resp.getStatusLine().getStatusCode());
			resp.close();
		}
		executorSerivce.shutdown();
	}

}

This code works perfectly with reordering of processing of the requests which can happen in multi-threading.