Skip to content

Unexpected behavior for: 5.9.1 Dirent: SecondaryFiles placed in different directories #2131

Closed
@mapk-amazon

Description

@mapk-amazon

My understanding of 5.9.1 Dirent in InitialWorkDirRequirement (https://www.commonwl.org/v1.2/CommandLineTool.html#Dirent) is to have all necessary files in the same directory. It behaves this way for files classes, but their secondary files are stored in a different directory.

Minimal example:

my-example.cwl:

cwlVersion: v1.2
class: CommandLineTool

requirements:
- class: InlineJavascriptRequirement
- class: ShellCommandRequirement
- class: InitialWorkDirRequirement
  listing:
      - entry: $(inputs.example_files)
        writable: true
inputs:
  example_files:
    type: File
    secondaryFiles:
    - pattern: "^.bye"
      required: true
outputs: 
  same_file:
    type: stdout

baseCommand: cat
arguments: 
  - $(inputs.example_files.path)
  - $(inputs.example_files.path.split('.')[0]+'.bye')

my-input.json

{
    "example_files": {
        "class": "File",
        "location": "primary_file.hi",
        "secondaryFiles": [
            {
                 "class": "File",
                "location": "primary_file.bye"
            }
        ]
    }
}

and I have two files as input primary_file.hi and primary_file.bye which are simple text files. It results in the error

INFO [job secondary-file-test.cwl] /tmp/yuwwd3k9$ /bin/sh \
    -c \
    cat /tmp/yuwwd3k9/primary_file.hi /tmp/yuwwd3k9/primary_file.bye > /tmp/yuwwd3k9/ddc706d0a33e29ca202ad17ea3b0fc5a517d8e13
cat: /tmp/yuwwd3k9/primary_file.bye: No such file or directory

where in debug mode we can see that the files are in different directories:

DEBUG [job secondary-file-test.cwl] initial work dir {
    "file:///home/--censored--/cwl-wf/cwl-for-cache/primary_file.hi": [
        "/home/--censored--/cwl-wf/cwl-for-cache/primary_file.hi",
        "/tmp/yuwwd3k9/primary_file.hi",
        "WritableFile",
        true
    ],
    "file:///home/--censored--/cwl-wf/cwl-for-cache/secondarydirectory/primary_file.bye": [
        "/home/--censored--/cwl-wf/cwl-for-cache/secondarydirectory/primary_file.bye",
        "/tmp/6qva1xjh/stge136a4ec-90a5-4e25-8d4b-6126bbb43292/primary_file.bye",
        "WritableFile",
        true
    ]
}

Is this the expected behavior? If yes, how could I enforce the secondary files are placed next to the primary file? In my use case its a genomics file and some indexes.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions