Jackson 2.8.x dependency problem w/ Elasticsearch 2.3.x by managed dependencies of Spring Boot 1.4.0

[agebhar1]

After upgrade from Spring Boot v1.3.5 to v1.4.0 with a embedded Elasticsearch node the application is broken by NoSuchMethodError:

java.lang.NoSuchMethodError: com.fasterxml.jackson.core.base.GeneratorBase.getOutputContext()Lcom/fasterxml/jackson/core/json/JsonWriteContext;
        at org.elasticsearch.common.xcontent.json.JsonXContentGenerator.writeEndRaw(JsonXContentGenerator.java:327) ~[elasticsearch-2.3.4.jar:2.3.4]
        at org.elasticsearch.common.xcontent.json.JsonXContentGenerator.writeRawField(JsonXContentGenerator.java:368) ~[elasticsearch-2.3.4.jar:2.3.4]
        at org.elasticsearch.common.xcontent.XContentBuilder.rawField(XContentBuilder.java:914) ~[elasticsearch-2.3.4.jar:2.3.4]
        at org.elasticsearch.common.xcontent.XContentHelper.writeRawField(XContentHelper.java:378) ~[elasticsearch-2.3.4.jar:2.3.4]
        at org.elasticsearch.search.internal.InternalSearchHit.toXContent(InternalSearchHit.java:476) ~[elasticsearch-2.3.4.jar:2.3.4]
        at org.elasticsearch.search.internal.InternalSearchHits.toXContent(InternalSearchHits.java:184) ~[elasticsearch-2.3.4.jar:2.3.4]
        at org.elasticsearch.search.internal.InternalSearchResponse.toXContent(InternalSearchResponse.java:111) ~[elasticsearch-2.3.4.jar:2.3.4]
        at org.elasticsearch.action.search.SearchResponse.toXContent(SearchResponse.java:195) ~[elasticsearch-2.3.4.jar:2.3.4]
        at org.elasticsearch.rest.action.support.RestStatusToXContentListener.buildResponse(RestStatusToXContentListener.java:43) ~[elasticsearch-2.3.4.jar:2.3.4]
        at org.elasticsearch.rest.action.support.RestStatusToXContentListener.buildResponse(RestStatusToXContentListener.java:38) ~[elasticsearch-2.3.4.jar:2.3.4]
        at org.elasticsearch.rest.action.support.RestStatusToXContentListener.buildResponse(RestStatusToXContentListener.java:30) ~[elasticsearch-2.3.4.jar:2.3.4]
        at org.elasticsearch.rest.action.support.RestResponseListener.processResponse(RestResponseListener.java:43) ~[elasticsearch-2.3.4.jar:2.3.4]
        at org.elasticsearch.rest.action.support.RestActionListener.onResponse(RestActionListener.java:49) ~[elasticsearch-2.3.4.jar:2.3.4]
        at org.elasticsearch.action.support.TransportAction$1.onResponse(TransportAction.java:89) [elasticsearch-2.3.4.jar:2.3.4]
        at org.elasticsearch.action.support.TransportAction$1.onResponse(TransportAction.java:85) [elasticsearch-2.3.4.jar:2.3.4]
        at org.elasticsearch.action.search.SearchQueryThenFetchAsyncAction$2.doRun(SearchQueryThenFetchAsyncAction.java:138) [elasticsearch-2.3.4.jar:2.3.4]
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-2.3.4.jar:2.3.4]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_101]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_101]
        at java.lang.Thread.run(Thread.java:745) [na:1.8.0_101]

The problematic code from Elasticsearch (JsonXContentGenerator:327)

public void writeEndRaw() {
    assert base != null : "JsonGenerator should be of instance GeneratorBase but was: " + generator.getClass();
    if (base != null) {
        base.getOutputContext().writeValue();
    }
}

Elasticsearch (v2.3.4) depends on Jackson v2.6.6 and is managed by Spring Boots provided version v2.8.1. In Jackson's GeneratorBase v2.6 the return type is JsonWriteContext

JsonWriteContext | getOutputContext()
Note: co-variant return type.

which has a method writeValue() but in GeneratorBase v2.8 the return type is changed to it's super type JsonStreamContext

JsonStreamContext | getOutputContext()
Note: type was co-variant until Jackson 2.7; reverted back to base type in 2.8 to allow for overriding by subtypes that use custom context type.

The current provided version of Jackson breaks usage of Elasticsearch with out-of-the-box Spring Boots 1.4 managed dependencies.

[agebhar1]

added a simple test case agebhar1/spring-boot-6508-jackson-elasticsearch

[anand1st]

This affects jest-2.0.3 functionality integration with spring-boot-starter-data-elasticsearch as well

[anand1st]

This problem is seen only in the embedded elasticsearch scenario. When used against a standalone elasticsearch, this is not a problem. In my case, I use the embedded elasticsearch for unit testing. One temporary workaround for me was to hardcode the jackson dependencies to version 2.6.6 and change the scope to test so that these older jackson libraries are not transitively passed down.

[jloisel]

This issue also happens with standalone elasticsearch when using elasticsearch java library as client. Downgrading Jackson to version 2.7.5 fixes the issue. This is not a Spring boot issue, it's elasticsearch which should upgrade to Jackson 2.8.x.

[agebhar1]

@jloisel 😕 the latest version of Elasticsearch 2.3.5 uses Jackson 2.6.6 ... as mentioned in #6536 the upcoming Elasticsearch version 5 seems to be using Jackson 2.8.1

[rburawes]

I have the same issue, I'm using spring-data-elasticsearch-2.0.2.RELEASE. I've downgraded my Jackson dependency to 2.7.6 and it solved the issue, at least for now.

[philwebb]

We're stuck between a rock and a hard place with this one. I don't think we want to downgrade our Jackson version just for Elasticsearch but we should document that a downgrade is necessary.

[philwebb]

Perhaps we could also throw a nicer error if we detect an invalid combination.

[agebhar1]

@philwebb Did you have something in mind to accomplish/detect it?