Skip to content

docker_workspace: support dockerignore to filter files from the workspace #401

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from

Conversation

d4l3k
Copy link
Member

@d4l3k d4l3k commented Feb 25, 2022

This adds a filter when building the docker workspace context to filter files specified in the .dockerignore file. This reduces the size of the patch when operating in a larger repo.

Reference:
https://docs.docker.com/engine/reference/builder/#dockerignore-file

Next steps from this would be to extend this logic to be workspace agnostic so we could have a .torchxignore file but not sure if that makes sense or is strictly necessary yet.

Test plan:

$  torchx run --scheduler aws_batch --wait --log dist.ddp --script foo.py  -j 1x1
$ pytest torchx/workspace/

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Feb 25, 2022
@facebook-github-bot
Copy link
Contributor

@d4l3k has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@codecov
Copy link

codecov bot commented Feb 25, 2022

Codecov Report

Merging #401 (6128afe) into main (8a7a95a) will increase coverage by 0.03%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #401      +/-   ##
==========================================
+ Coverage   94.50%   94.53%   +0.03%     
==========================================
  Files          66       66              
  Lines        3638     3663      +25     
==========================================
+ Hits         3438     3463      +25     
  Misses        200      200              
Impacted Files Coverage Δ
torchx/workspace/docker_workspace.py 95.19% <100.00%> (+1.52%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8a7a95a...6128afe. Read the comment docs.

@facebook-github-bot
Copy link
Contributor

@d4l3k has updated the pull request. You must reimport the pull request before landing.

@facebook-github-bot
Copy link
Contributor

@d4l3k has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@d4l3k d4l3k deleted the dockerignore branch February 28, 2022 21:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants