Thrown exceptions in FallbackFactory are wrapped with IllegalStateException

[shellrausch]

Reproducable with: Kotlin and Java
Java version: 11
Spring Boot version: 2.6.3
Spring Cloud version: 2021.0.0
Client: OpenFeign

I am migrating one of my spring boot projects from hystrix to r4j and noticed a difference in the exception propagation in comparision to hystrix. I really don't know whether this is an intended behaviour, nor whether it belongs to openfeign, r4j or spring :(.

My FallbackFactory throws my CustomerNotFoundException:

@Component
class CustomerClientFallbackFactory : FallbackFactory<CustomerClient> {
    override fun create(cause: Throwable): CustomerClient =
        object : CustomerClient {
            override fun test() {
                // Why will this exception end up in an IllegalStateException?
                throw CustomerNotFoundException()
            }
        }
}

class CustomerNotFoundException : BusinessException("not found")

My feign client uses this FallbackFactory:

@FeignClient(
    name = "customer",
    url = "http://localhost:8888",
    configuration = [CustomerClientConfig::class],
    fallbackFactory = CustomerClientFallbackFactory::class
)
interface CustomerClient {
    @RequestLine("GET /")
    fun test()
}

To simulate and illustrate the behaviour I have created two endpoints with the same client:

@RestController
class CustomerController(
    private val customerClient: CustomerClient
) {
    private val log = KotlinLogging.logger {}

    @GetMapping("/wrap")
    fun wrap() {
        try {
            customerClient.test()
        } catch (e: Exception) {
            log.info { "Exception: " + e.javaClass }
            // Output: Exception: class java.lang.IllegalStateException
            log.info(e) {}
        }
    }

    @GetMapping("/unwrap")
    fun unwrap() {
        try {
            unwrapFeignCircuitBreakerExceptions {
                customerClient.test()
            }
        } catch (e: Exception) {
            log.info { "Exception: " + e.javaClass }
            // Output: Exception: class org.example.auth.CustomerNotFoundException
        }
    }
}

My expectation is that wrap() catches CustomerNotFoundException - but this is not the case. Instead I get a IllegalStateException which wraps my CustomerNotFoundException:

java.lang.IllegalStateException: java.lang.reflect.InvocationTargetException
	at org.springframework.cloud.openfeign.FeignCircuitBreakerInvocationHandler.lambda$invoke$0(FeignCircuitBreakerInvocationHandler.java:99) ~[spring-cloud-openfeign-core-3.1.0.jar:3.1.0]
	at io.vavr.control.Try.lambda$recover$6ea7267f$1(Try.java:949) ~[vavr-0.10.2.jar:na]
	at io.vavr.control.Try.of(Try.java:75) ~[vavr-0.10.2.jar:na]
	at io.vavr.control.Try.recover(Try.java:949) ~[vavr-0.10.2.jar:na]
	at org.springframework.cloud.circuitbreaker.resilience4j.Resilience4JCircuitBreaker.run(Resilience4JCircuitBreaker.java:123) ~[spring-cloud-circuitbreaker-resilience4j-2.1.0.jar:2.1.0]
	at org.springframework.cloud.openfeign.FeignCircuitBreakerInvocationHandler.invoke(FeignCircuitBreakerInvocationHandler.java:102) ~[spring-cloud-openfeign-core-3.1.0.jar:3.1.0]
	at com.sun.proxy.$Proxy77.test(Unknown Source) ~[na:na]
	at org.example.auth.CustomerController.wrap(CustomerController.kt:18) ~[main/:na]
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:na]
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
	at java.base/java.lang.reflect.Method.invoke(Method.java:566) ~[na:na]
	at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205) ~[spring-web-5.3.15.jar:5.3.15]
	at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:150) ~[spring-web-5.3.15.jar:5.3.15]
	at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:117) ~[spring-webmvc-5.3.15.jar:5.3.15]
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:895) ~[spring-webmvc-5.3.15.jar:5.3.15]
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:808) ~[spring-webmvc-5.3.15.jar:5.3.15]
	at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87) ~[spring-webmvc-5.3.15.jar:5.3.15]
	at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1067) ~[spring-webmvc-5.3.15.jar:5.3.15]
	at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:963) ~[spring-webmvc-5.3.15.jar:5.3.15]
	at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006) ~[spring-webmvc-5.3.15.jar:5.3.15]
	at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:898) ~[spring-webmvc-5.3.15.jar:5.3.15]
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:655) ~[tomcat-embed-core-9.0.56.jar:4.0.FR]
	at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883) ~[spring-webmvc-5.3.15.jar:5.3.15]
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:764) ~[tomcat-embed-core-9.0.56.jar:4.0.FR]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:227) ~[tomcat-embed-core-9.0.56.jar:9.0.56]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) ~[tomcat-embed-core-9.0.56.jar:9.0.56]
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53) ~[tomcat-embed-websocket-9.0.56.jar:9.0.56]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) ~[tomcat-embed-core-9.0.56.jar:9.0.56]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) ~[tomcat-embed-core-9.0.56.jar:9.0.56]
	at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100) ~[spring-web-5.3.15.jar:5.3.15]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) ~[spring-web-5.3.15.jar:5.3.15]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) ~[tomcat-embed-core-9.0.56.jar:9.0.56]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) ~[tomcat-embed-core-9.0.56.jar:9.0.56]
	at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93) ~[spring-web-5.3.15.jar:5.3.15]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) ~[spring-web-5.3.15.jar:5.3.15]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) ~[tomcat-embed-core-9.0.56.jar:9.0.56]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) ~[tomcat-embed-core-9.0.56.jar:9.0.56]
	at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201) ~[spring-web-5.3.15.jar:5.3.15]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) ~[spring-web-5.3.15.jar:5.3.15]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) ~[tomcat-embed-core-9.0.56.jar:9.0.56]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) ~[tomcat-embed-core-9.0.56.jar:9.0.56]
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:197) ~[tomcat-embed-core-9.0.56.jar:9.0.56]
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97) ~[tomcat-embed-core-9.0.56.jar:9.0.56]
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:540) ~[tomcat-embed-core-9.0.56.jar:9.0.56]
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:135) ~[tomcat-embed-core-9.0.56.jar:9.0.56]
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92) ~[tomcat-embed-core-9.0.56.jar:9.0.56]
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78) ~[tomcat-embed-core-9.0.56.jar:9.0.56]
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:357) ~[tomcat-embed-core-9.0.56.jar:9.0.56]
	at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:382) ~[tomcat-embed-core-9.0.56.jar:9.0.56]
	at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65) ~[tomcat-embed-core-9.0.56.jar:9.0.56]
	at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:895) ~[tomcat-embed-core-9.0.56.jar:9.0.56]
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1732) ~[tomcat-embed-core-9.0.56.jar:9.0.56]
	at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) ~[tomcat-embed-core-9.0.56.jar:9.0.56]
	at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191) ~[tomcat-embed-core-9.0.56.jar:9.0.56]
	at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659) ~[tomcat-embed-core-9.0.56.jar:9.0.56]
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) ~[tomcat-embed-core-9.0.56.jar:9.0.56]
	at java.base/java.lang.Thread.run(Thread.java:829) ~[na:na]
Caused by: java.lang.reflect.InvocationTargetException: null
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:na]
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
	at java.base/java.lang.reflect.Method.invoke(Method.java:566) ~[na:na]
	at org.springframework.cloud.openfeign.FeignCircuitBreakerInvocationHandler.lambda$invoke$0(FeignCircuitBreakerInvocationHandler.java:96) ~[spring-cloud-openfeign-core-3.1.0.jar:3.1.0]
	... 57 common frames omitted
Caused by: org.example.auth.CustomerNotFoundException: not found
	at org.example.auth.CustomerClientFallbackFactory$create$1.test(CustomerClientFallbackFactory.kt:12) ~[main/:na]
	... 62 common frames omitted

unwrap() uses an infamous lambda unwrapFeignCircuitBreakerExceptions (found on the internet) which unwraps the exception chain and gives me the desired result. Which is of course far from ideal because I have to wrap every client call with unnecessary boilerplate code. Furthermore it looks like a hack or workaround:

inline fun <T> unwrapFeignCircuitBreakerExceptions(callable: () -> T): T {
    return try {
        callable.invoke()
    } catch (ex: InvocationTargetException) {
        throw ex.cause ?: ex
    } catch (ex: NoFallbackAvailableException) {
        throw ex.cause ?: ex
    } catch (ex: IllegalStateException) {
        throw (ex.cause?.cause ?: ex.cause) ?: ex
    }
}

Is there an official way to handle/configure this? I mean it is a valid use-case to throw an exception in some fallback cases instead of providing a fallback object. Or is this a bug - as this behaviour was not present in combination with hystrix?

Minimal, complete, verifiable example project: https://github.com/shellrausch/feign-r4j-fallback

Thanks!

[OlgaMaciaszek]

Hello, @shellrausch - thanks for submitting this issue. I was able to reproduce it. I do not think it's a very good practice to throw an exception in the fallback, however, I also don't think we should throw a wrapped exception if it does get done. It was probably added in the first place to handle unchecked exceptions. Have added an unwrapping method on a branch: https://github.com/spring-cloud/spring-cloud-openfeign/tree/unwrap-cb-exception. Will add tests tomorrow and merge.

[agav]

Hi, thx for the fix. When it is planned to be released?

[OlgaMaciaszek]

Should be released before the end of April.

[hingbong]

when I do not use fallback, it doesn't unwrap execption