WDL needs a fail function

[markjschreiber]

It would be useful for a WDL script to be able to indicate to the engine that something has happened that requires the engine to fail the workflow. For example:

    input {
        Array[Float] downsample_fractions
    }

    # later ...

    scatter (fraction in downsample_fractions) {
        if (fraction <= 0.0 || fraction >= 1.0) {
            fail("all downsample fractions must be greater than 0 and less than 1")
        }
    }

Although the validation could be done by a task this would be very heavy weight for such a simple check. I would propose two forms of the fail function:

1. fail(String) - The engine will fail the workflow and emit the error message. Any tasks still running when the fail call is made will keep running but no new tasks will start. The reason to keep running tasks running is mostly for call caching/ resume purposes. Essentially a graceful fail.
2. fail(String, Boolean) - Same as above but if the Boolean is true then the engine will halt and fail any running tasks and fail and exit the workflow as soon as possible.

[markjschreiber]

A potential complication is that task calls after the potential fail may need to wait before starting. Kind of like an implicit pre-condition has been set on them.

[geoffjentry]

FWIW there's soime ancient, pre-OpenWDL discussion (complete with the same poor hack that was discussed in slack!)

[markjschreiber]

FWIW there's soime ancient, pre-OpenWDL discussion (complete with the same poor hack that was discussed in slack!)

Interesting. And the request is only 8 years old so that's encouraging 😆

[geoffjentry]

Yeah when I saw it earlier I knew it sounded familiar. I have vague memories of discussing it during standup & such way back when. My assumption is it just got shoved way down the priority list 🤷

[claymcleod]

I think this makes sense to add. There's no real reason that you should have to call a bash task to exit the workflow. My feeling is that there needs to be two functions for this kind of thing.

• error(String[, Int]) takes a string, prints it to stderr, and exits the process. By default, it exits with code 1, but the exit code can be specified.
• warn(String) takes a string and prints it as a warning out to stderr.

[geoffjentry]

I would suggest sticking with the general model of starting simple and then building up if and when there's a demonstrated need. This topic has come up a few times over the years (e.g. the old link I put up). The ask has always bene pretty basic: an easy way to fail a workflow without needing to call a task. In general my mental picture has always been similar to what @markjschreiber posted here. I don't love the 2nd variant he has: IMO breadth-first vs depth-first traversal of the DAG should be configurable by the user not the workflow author.

[geoffjentry]

@claymcleod By default, it exits with code 1, but the exit code can be specified. - not sure what you mean here. There's not an exit code for a WDL workflow. To what process are you referring here?