Conditional (Java configured) steps cannot be scoped [BATCH-2747]

[spring-projects-issues]

Mattias Jiderhamn opened BATCH-2747 and commented

Configuring conditional steps using Java config (XML untested) fails if any of the steps is scoped using Job scope or Step scope.

To reproduce, use the example at https://docs.spring.io/spring-batch/4.0.x/reference/html/step.html#controllingStepFlow, i.e.

@Bean
public Job job() {
        return this.jobBuilderFactory.get("job")
                                .start(stepA())
                                .on("*").to(stepB())
                                .from(stepA()).on("FAILED").to(stepC())
                                .end()
                                .build();
}

and then add @JobScope or @StepScope on stepA(), stepB() or stepC().

The underlying reason is that to configure a flow step, the framework wants to get the steps name using getName(), at which point the bean is resolved and the scope inferred.

Stack trace (4.0.1)

Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'myJob' defined in se.jiderhamn.ScopeTestConfiguration: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.batch.core.Job]: Factory method 'parseCallLogJob' threw exception; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'scopedTarget.stepA': Scope 'job' is not active for the current thread; consider defining a scoped proxy for this bean if you intend to refer to it from a singleton; nested exception is java.lang.IllegalStateException: No context holder available for job scope
	at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:583)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1249)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1098)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:545)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:502)
	at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:312)
	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:228)
	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:310)
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:200)
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:756)
	at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:868)
	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:549)
	at org.springframework.test.context.support.AbstractGenericContextLoader.loadContext(AbstractGenericContextLoader.java:128)
	at org.springframework.test.context.support.AbstractGenericContextLoader.loadContext(AbstractGenericContextLoader.java:60)
	at org.springframework.test.context.support.AbstractDelegatingSmartContextLoader.delegateLoading(AbstractDelegatingSmartContextLoader.java:109)
	at org.springframework.test.context.support.AbstractDelegatingSmartContextLoader.loadContext(AbstractDelegatingSmartContextLoader.java:246)
	at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContextInternal(DefaultCacheAwareContextLoaderDelegate.java:99)
	at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContext(DefaultCacheAwareContextLoaderDelegate.java:117)
	... 24 more
Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.batch.core.Job]: Factory method 'parseCallLogJob' threw exception; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'scopedTarget.stepA': Scope 'job' is not active for the current thread; consider defining a scoped proxy for this bean if you intend to refer to it from a singleton; nested exception is java.lang.IllegalStateException: No context holder available for job scope
	at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:186)
	at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:575)
	... 41 more
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'scopedTarget.stepA': Scope 'job' is not active for the current thread; consider defining a scoped proxy for this bean if you intend to refer to it from a singleton; nested exception is java.lang.IllegalStateException: No context holder available for job scope
	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:357)
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:200)
	at org.springframework.aop.target.SimpleBeanTargetSource.getTarget(SimpleBeanTargetSource.java:35)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:193)
	at com.sun.proxy.$Proxy32.getName(Unknown Source)
	at org.springframework.batch.core.job.builder.FlowBuilder.createState(FlowBuilder.java:282)
	at org.springframework.batch.core.job.builder.FlowBuilder.doStart(FlowBuilder.java:265)
	at org.springframework.batch.core.job.builder.FlowBuilder.start(FlowBuilder.java:122)
	at org.springframework.batch.core.job.builder.JobFlowBuilder.<init>(JobFlowBuilder.java:39)
	at org.springframework.batch.core.job.builder.SimpleJobBuilder.on(SimpleJobBuilder.java:91)
	at se.jiderhamn.ScopeTestConfiguration.parseCallLogJob(ScopeTestConfiguration.java:36)
	at se.jiderhamn.ScopeTestConfiguration$$EnhancerBySpringCGLIB$$240ba886.CGLIB$parseCallLogJob$3(<generated>)
	at se.jiderhamn.ScopeTestConfiguration$$EnhancerBySpringCGLIB$$240ba886$$FastClassBySpringCGLIB$$faf3ccf4.invoke(<generated>)
	at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
	at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:361)
	at se.jiderhamn.ScopeTestConfiguration$$EnhancerBySpringCGLIB$$240ba886.parseCallLogJob(<generated>)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:155)
	... 42 more
Caused by: java.lang.IllegalStateException: No context holder available for job scope
	at org.springframework.batch.core.scope.JobScope.getContext(JobScope.java:159)
	at org.springframework.batch.core.scope.JobScope.get(JobScope.java:92)
	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:345)
	... 62 more

---

Affects: 4.0.1, 4.1.0.M2

Reference URL: https://stackoverflow.com/questions/55571210/

2 votes, 4 watchers

[spring-projects-issues]

Michael Minella commented

What is the use case where the beans must be scoped?

[spring-projects-issues]

Mattias Jiderhamn commented

Michael Minella, the most obvious use case would be when JobParameters are used in the step definition

@Bean
@JobScope
Step readDataFromFile(@Value("#{jobParameters[filePath]}") String filePath) {
...
}

 

There does however seem to be at least one workaround, by separately defining the ItemReader/Processor/Writer as @JobScope:d beans like so

@Bean
@JobScope
ItemReader reader(@Value("#{jobParameters[filePath]}") String filePath) {
...
}

@Bean(name = "stepA")
Step readDataFromFile(ItemReader reader) {
...
}

@Bean
Job myJob(@Qualifier("stepA") stepA) {
        return this.jobBuilderFactory.get("job")
                                .start(stepA)
                                .on("*").to(stepB())
                                .from(stepA).on("FAILED").to(stepC())
                                .end()
                                .build();
}

[spring-projects-issues]

ArtyomGabeev commented

Hi,

I think issue is related to FlowBuilder - it's impossible to use scoped steps/flows inside any other flow.

Here is a small example:

@Configuration
public class JobWithFlowConfig {

    @Bean
    public Job job(JobBuilderFactory jobBuilderFactory, Flow flow) {
        return jobBuilderFactory.get("job")
                .start(flow)
                .end()
                .build();
    }

    @Bean
    public Flow flow(Step step) {
        return new FlowBuilder<Flow>("flow")
                .from(step)
                .end();
    }

    @Bean
    @JobScope
    public Step step(StepBuilderFactory stepBuilderFactory, @Value("#{jobParameters['test']}") String test) {
        return stepBuilderFactory.get("step")
                .tasklet(((contribution, chunkContext) -> {
                    System.out.println(test);
                    return RepeatStatus.FINISHED;
                }))
                .allowStartIfComplete(true)
                .build();
    }

}

 

Issue happens in FlowBuilder.createState - calling getName on a step/flow causes lazy proxy initialization, but there is no Job/Step scope at this time.

In order to fix it, I suggest reuse logic of conters naming for step/flow:

...
private int stepCounter = 0;

private int flowCounter = 0;
...

private State createState(Object input) {
...
states.put(input, new StepState(prefix + "step" + (stepCounter++), step)); //remove getName call
...
states.put(input, new FlowState((Flow) input, prefix + "flow" + (flowCounter++))); //same for the flow
}

Thanks,
Artyom

[spring-projects-issues]

ArtyomGabeev commented

Regarding original issue: calling on over jobBuilder, simply converts previous steps into FlowBuilder, that explains the behaviour.
It can be observed though the stack trace:

at com.sun.proxy.$Proxy32.getName(Unknown Source)
	at org.springframework.batch.core.job.builder.FlowBuilder.createState(FlowBuilder.java:282)
	at org.springframework.batch.core.job.builder.FlowBuilder.doStart(FlowBuilder.java:265)
	at org.springframework.batch.core.job.builder.FlowBuilder.start(FlowBuilder.java:122)
	at org.springframework.batch.core.job.builder.JobFlowBuilder.<init>(JobFlowBuilder.java:39)
	at org.springframework.batch.core.job.builder.SimpleJobBuilder.on(SimpleJobBuilder.java:91)

[spring-projects-issues]

Michael Minella commented

This still doesn't make sense to me.  You need to be in the context of a step in order to to use step scope.  What you are trying to do is causing premature initialization.  I don't see this as a bug but works as designed.

[spring-projects-issues]

ArtyomGabeev commented

What about JobScope?

In general step can be annotated with @JobScope in order to have access to jobParameters.

 

Michael Minella, you right, it was my mistake when I mentioned StepScope in this issue, because it doesn't make sense.

But this fix is still reasonable if you want to create a JobScoped step, and pass some parameters (like chunk size for example)

 

BTW: original author has an issue with JobScoped step, according to stack trace.

[spring-projects-issues]

Michael Minella commented

This really also works as designed.  There must be a running job in order for the proxy to be resolved.  The steps are created before the job is executed so the idea that you can use job scope on a step still doesn't apply.  I'd argue that we may need to be more specific around our documentation, but you do not have a running job when a step is configured so there is no way for job scope to work on the configuration of a step as you are attempting to do.

[spring-projects-issues]

ArtyomGabeev commented

Thanks, now I see your point. My initial understanding was that JobScope can be applied on anything except Job.

So there is no way to bind jobParameters to step configuration, so let's say I can not adjust step chunk size based on jobParameters? 

Another use case is to configure gridSize based on jobParameters.

[spring-projects-issues]

ArtyomGabeev commented

Also, it's possible to configure JobScope'd step, if there is no flow:

@Configuration
public class JobConfig {

    @Bean
    public Job job(JobBuilderFactory jobBuilderFactory, Step step) {
        return jobBuilderFactory.get("job")
                .start(step)
                .build();
    }
    
    @Bean
    @JobScope
    public Step step(StepBuilderFactory stepBuilderFactory, @Value("#{jobParameters['test']}") String test) {
        return stepBuilderFactory.get("step")
                .tasklet(((contribution, chunkContext) -> {
                    System.out.println(test);
                    return RepeatStatus.FINISHED;
                }))
                .allowStartIfComplete(true)
                .build();
    }

} 

This configuration works fine. Also I want to try same two configurations in XML to confirm behaviour.

[spring-projects-issues]

ArtyomGabeev commented

Michael Minella

Finally, I agree with that step can not be scoped, when I get to the partitioned steps (there is no job scope).

Probably documentation should be more specific in terms what can be proxied.

Thank you for clarification.

 

[spring-projects-issues]

Mattias Jiderhamn commented

Sorry for being late back to the party.

Even though it may "work as designed", at least personally I think that design is very far from intuitive and definitely could use clarifying in the documentation. I spent quite some time figuring out a working configuration using trial and error.

 

The steps are created before the job is executed so the idea that you can use job scope on a step still doesn't apply.

+Despite having read the documentation+ this really doesn't make sense to me. If you create a job scoped or step scoped bean - such as a reader, as in my example from September 1st - you may use that bean when configuring your step. But you may not make the step itself job scoped nor step scoped - am I right? This seems like saying steps may be implicitly job scoped or step scoped (due to the dependency on the scoped bean), but not explicitly. That is - with the current documentation - counter-intuitive. I mean, what are the job and step scopes for, if not for being used in the configuration???

 

Here is a variation of my September 1st example. The fact that it also works for me only adds to the confusion.

@Bean
@JobScope
ItemReader reader(@Value("#{jobParameters[filePath]}") String filePath) {
...
}

@Bean
Step readDataFromFile(ItemReader reader) {
  return steps.get("readDataFromFileStep")
        .<X, Y>chunk(100)
        .reader(reader("Overridden by expression"))
        ...
}

(Sorry for spamming you with updates fixing Jira copy/paste issues)