Remote partitioning with aggregate replies, the partition manager didn't notify the completion correctly if the manager app got restarted

[sereysopheak]

Remote partitioning with aggregate replies, the partition manager didn't notify the job completion correctly if the manager app got restarted. I'm trying to configure it to use Kafka as the messaging channel. Everything is working fine if the manager keeps running while worker processing the job. At one point if manager instance is shut down and restarted, I can see the worker is still processing the job normally and replies are sent from the worker, but the manager itself didn't notify the job completion correctly even though the replies messages were consumed. And the status is stuck in STARTED with exit code UNKNOWN

Environment
Spring Batch version: 4.2.7
Java 11
database: MySQL

Steps to reproduce

• Running the manager and worker on different instance
• While worker is executing the job, restart the manager
• Manager never update the status to complete/failed and the after job listener never got triggered.

Expected behavior
As it's done using the replies through messaging event, manager should be able to consume the replies message and reconcile the job status.

[fmbenhassine]

Thank you for opening this issue. This should not happen if replies from workers are still stored in the message broker when the manager dies. When restarted, the manager should be able to consume those messages and reconcile the job status.

We do not exclude that this could be a bug in Spring Batch. However, we would like to validate that with a minimal complete verifiable example. Could you please take some time to create a minimal example that reproduces the problem? To help you in reporting your issue, we have prepared a project template that you can use as a starting point. Please check the Issue Reporting Guidelines for more details about this. You can also find a sample of remote partitioning with message aggregation here.

Thank you for your collaboration.

[fracoder]

Hi,
I am experiencing a similar issue but on a different scenario. In my case the job is configured in pull mode and the updates from worker are sourced from DB. When master dies there is no way to recover the result of workers in complete state on a subsequent Job restart. Looking at the code looks like aggregator is only receiving results from the steps executed in the current run, those related to previous run and in complete status are not aggregated because, as expected, are not executed.
A potential solution could be recovering the step execution contexts from job repository but the expectation is master recovering the job/step status automatically. Do you have any clue?

Thanks,
Francesco