Elasticsearch Operation via REST API using Apache HttpClient in Java

Previously, I tested some very basic operations in Elasticsearch via REST API. So, in this post, I am going to implement those very basic operations in Java with Apache HttpClient.

Import modules

Basic Elasticsearch operations using HTTP GET and POST methods are used in this post. So you need to import the following modules.

import org.apache.commons.codec.binary.Base64;
import org.apache.http.HttpHeaders;
import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.client.ResponseHandler;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.entity.StringEntity;
import org.apache.http.impl.client.BasicResponseHandler;
import org.apache.http.impl.client.DefaultHttpClient;

Send GET HTTP Request

In this section, an implementation in Java of the following Elasticsearch operation is explained.

root@ubuntu:~# curl -XGET "http://127.0.0.1:9200/"

First of all, you need to define HttpClient and HttpGet objects.

        HttpClient client = new DefaultHttpClient();
        HttpGet request = new HttpGet(base_url);

Next, you need to add auth header to the request object.

        String auth = elastic_user + ":" + elastic_password;
        byte[] encodedAuth = Base64.encodeBase64(
                auth.getBytes(Charset.forName("ISO-8859-1")));
        String authHeader = "Basic " + new String(encodedAuth);
        request.setHeader(HttpHeaders.AUTHORIZATION, authHeader);

Then, you can send the GET Request and the response will return.

        ResponseHandler<String> responseHandler = new BasicResponseHandler();
        String response = client.execute(request, responseHandler);
        logger.info("Sending 'GET' request to URL : " + base_url);
        logger.info("HTTP Response: " + response);

In String response, only the body of response is contained like below.

2017-08-27 03:51:46,538 [main] INFO : HTTP Response: {
  "name" : "MBUCDnj",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "3QGMhCBPQSSNusOn6m325g",
  "version" : {
    "number" : "5.5.2",
    "build_hash" : "b2f0c09",
    "build_date" : "2017-08-14T12:33:14.154Z",
    "build_snapshot" : false,
    "lucene_version" : "6.6.0"
  },
  "tagline" : "You Know, for Search"
}

Memo

As Elasticsearch returns its response in Json format, I tried to find a way to flatten its json response in Java code like I did in Scala. Honestly, I struggled to implement the function in scala, but I found a very easy way to do the same in Java just by googling like java flatten json. You can use this library https://github.com/wnameless/json-flattener.

        Map<String, Object> flattenJson = JsonFlattener.flattenAsMap(response);
        logger.info("Flatten the response json: "+ flattenJson);

Then, you can get the response in flattened json format like following.

2017-08-27 03:51:46,615 [main] INFO : Flatten the response json: {"name":"MBUCDnj","cluster_name":"elasticsearch","cluster_uuid":"3QGMhCBPQSSNusOn6m325g","version.number":"5.5.2","version.build_hash":"b2f0c09","version.build_date":"2017-08-14T12:33:14.154Z","version.build_snapshot":false,"version.lucene_version":"6.6.0","tagline":"You Know, for Search"}

Send POST HTTP Request

In this section, an implementation in Java of the following Elasticsearch operation is explained.

root@ubuntu:~# curl -XPOST  "http://127.0.0.1:9200/test/user" -d '{"name":"test_user"}'

Like I did in GET method, you need to define HttpClient and HttpPost objects at first.
Note that you need to define url including index and type like http://url:port/index/type.

        HttpClient client = new DefaultHttpClient();
        HttpPost post_request = new HttpPost(url);

Next, you need to add auth header to the request object.

        String auth = elastic_user + ":" + elastic_password;
        byte[] encodedAuth = Base64.encodeBase64(
                auth.getBytes(Charset.forName("ISO-8859-1")));
        String authHeader = "Basic " + new String(encodedAuth);
        request.setHeader(HttpHeaders.AUTHORIZATION, authHeader);

Then, set Json data to post. In this test case, I just defined json_data as String.
Maybe in other cases, you will need to implement some json operation using such as gson and org.json. So far, I have just briefly googled it but gson seems a little bit easier to implement.

gson: https://github.com/google/gson
org.json: https://stleary.github.io/JSON-java/

        StringEntity json_to_post = new StringEntity(json_data);
        post_request.setEntity(json_to_post);

Finaly, execute HTTP POST Request.

    HttpResponse response = client.execute(post_request);

    logger.info("Response Code : " + response.getStatusLine().getStatusCode());
    BufferedReader rd = new BufferedReader(new InputStreamReader(response.getEntity().getContent()));
    StringBuffer result = new StringBuffer();
    String line = "";
        while ((line = rd.readLine()) != null) {
                result.append(line);
                }
                logger.info(result.toString());

You will get a response like this.

2017-08-27 03:51:46,661 [main] INFO : Response Code : 201
2017-08-27 03:51:46,661 [main] INFO : {"_index":"test","_type":"user","_id":"AV4jUaROJrPGAVUzWKwL","_version":1,"result":"created","_shards":{"total":2,"successful":1,"failed":0},"created":true}