Updating Kafka 1.0.1 -> Kafka 1.1.0 causes compilation error in Spring-Kafka-Test.

[nklmish]

Note: To reproduce this issue locally please clone this.

After updating Kafka to 1.0.1, Spring-Kafka-test (@KafkaEmbeddedTest no longer seems to work. It gives the following exception.

java.lang.IllegalStateException: Failed to load ApplicationContext

	at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContext(DefaultCacheAwareContextLoaderDelegate.java:125)
	at org.springframework.test.context.support.DefaultTestContext.getApplicationContext(DefaultTestContext.java:107)
	at org.springframework.test.context.support.DependencyInjectionTestExecutionListener.injectDependencies(DependencyInjectionTestExecutionListener.java:117)
	at org.springframework.test.context.support.DependencyInjectionTestExecutionListener.prepareTestInstance(DependencyInjectionTestExecutionListener.java:83)
	at org.springframework.boot.test.autoconfigure.SpringBootDependencyInjectionTestExecutionListener.prepareTestInstance(SpringBootDependencyInjectionTestExecutionListener.java:44)
	at org.springframework.test.context.TestContextManager.prepareTestInstance(TestContextManager.java:242)
	at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.createTest(SpringJUnit4ClassRunner.java:227)
	at org.springframework.test.context.junit4.SpringJUnit4ClassRunner$1.runReflectiveCall(SpringJUnit4ClassRunner.java:289)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.methodBlock(SpringJUnit4ClassRunner.java:291)
	at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:246)
	at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:97)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.springframework.test.context.junit4.statements.RunBeforeTestClassCallbacks.evaluate(RunBeforeTestClassCallbacks.java:61)
	at org.springframework.test.context.junit4.statements.RunAfterTestClassCallbacks.evaluate(RunAfterTestClassCallbacks.java:70)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(SpringJUnit4ClassRunner.java:190)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
	at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'kafkaEmbedded': Invocation of init method failed; nested exception is java.lang.NoSuchMethodError: kafka.utils.Logging.kafka$utils$Logging$_setter_$loggerName_$eq(Ljava/lang/String;)V
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1710)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:414)
	at org.springframework.kafka.test.context.EmbeddedKafkaContextCustomizer.customizeContext(EmbeddedKafkaContextCustomizer.java:107)
	at org.springframework.boot.test.context.SpringBootContextLoader$ContextCustomizerAdapter.initialize(SpringBootContextLoader.java:336)
	at org.springframework.boot.SpringApplication.applyInitializers(SpringApplication.java:626)
	at org.springframework.boot.SpringApplication.prepareContext(SpringApplication.java:366)
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:325)
	at org.springframework.boot.test.context.SpringBootContextLoader.loadContext(SpringBootContextLoader.java:138)
	at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContextInternal(DefaultCacheAwareContextLoaderDelegate.java:99)
	at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContext(DefaultCacheAwareContextLoaderDelegate.java:117)
	... 25 more
Caused by: java.lang.NoSuchMethodError: kafka.utils.Logging.kafka$utils$Logging$_setter_$loggerName_$eq(Ljava/lang/String;)V
	at kafka.utils.Logging$class.$init$(Logging.scala:23)
	at kafka.utils.TestUtils$.<init>(TestUtils.scala:68)
	at kafka.utils.TestUtils$.<clinit>(TestUtils.scala)
	at kafka.zk.EmbeddedZookeeper.<init>(EmbeddedZookeeper.scala:29)
	at org.springframework.kafka.test.rule.KafkaEmbedded.startZookeeper(KafkaEmbedded.java:318)
	at org.springframework.kafka.test.rule.KafkaEmbedded.before(KafkaEmbedded.java:185)
	at org.springframework.kafka.test.rule.KafkaEmbedded.afterPropertiesSet(KafkaEmbedded.java:180)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1769)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1706)
	... 34 more

If we try to exclude kafka dependency from org.springframework.kafka:spring-kafka-test:2.1.4.RELEASE and add compile ("org.apache.kafka:kafka_2.12:1.1.0:test") , then TestUtils.createBrokerConfig results in compilation error because of Api-change. Moreover, KIP-6317 removed Log4J, hence Log4JLevelAdjuster can't be compiled due to this and this.

@garyrussell Do we still need Log4jLevelAdjuster ?

I can also submit the PR with the fixes mentioned above.
Thanks.

[garyrussell]

We will be moving master to 2.2 after we release 2.1.5 later this week; at that time, we will upgrade to the 1.1.0 kafka-clients.

We will need to upgrade the tests to use Log4j2 and port the Log4j2LevelAdjuster from Spring Integration.

[garyrussell]

We will have to examine the API changes to see if we can support a 1.1.0 client in the 2.1.x line; perhaps by using reflection; we did something similar in 1.3.x to support the 1.0.0 client.

2.1.x has to maintain compatibility with 1.0.0.

[garyrussell]

Thanks for the offer of a PR but we have to maintain 1.0.x compatibility; see my PR, which will be merged before 2.1.5 is released.