dockerPull field in DockerRequirement should pull image updates before running

[adkinsrs]

I have been running some CWL workflows that utilize the DockerRequirement "dockerPull" field, and when the Docker image in question is not present locally, the field works as intended, pulling the image first before running. However, if the image is updated in the Docker registry sometime later, the "dockerPull" field will continue to run the local image with no actual attempt to do a "docker pull" for updates. This requires me to have to do a manual "docker pull" on the command line before invoking the CWL. Is it possible to have CWL perform an actual "docker pull" every time the workflow is invoked with the "dockerPull" field (to grab the latest image update if available), as opposed to simply doing a "docker run" if the image already exists locally?

[tetron]

Hi @adkinsrs

I think we could add a flag. It's not the default behavior because (a) it's not a good idea to change something on the system without explicit approval from the user and (b) it's not clear what the desired behavior should be if docker pull fails. Should it fail, or run anyway?

[adkinsrs]

I believe that adding a "pull updated image" flag would infer explicit approval from the user to update the local image, and I would be in favor of that. As for having cwltool fail or run with the local image, either method is fine by me. However, if I am electing to use this flag, it's because my current copy of the image is not sufficient for the task, either due to being broke/buggy or for some other reason. Thus I think having it fail may be better if the flag is set and the resulting "docker pull" does not work.

[kapilkd13]

Hi @tetron As you said, What should happen if the docker pull fails in this case. I can work on this issue once we decide that. Personally I am satisfied with @adkinsrs point, we should fail if docker pull doesn't work. What is your opinion on this.

[tetron]

I agree, if you requested it to pull, and the pull failed, that should be an error. The user still has the option of running without the pull flag, which will try to use the image already present.