Skip to content
This repository was archived by the owner on Jun 3, 2025. It is now read-only.

check file changed in loop#1302

Merged
tejal29 merged 6 commits intoGoogleContainerTools:masterfrom
tejal29:check_file_changed_in_loop
Jun 6, 2020
Merged

check file changed in loop#1302
tejal29 merged 6 commits intoGoogleContainerTools:masterfrom
tejal29:check_file_changed_in_loop

Conversation

@tejal29
Copy link
Copy Markdown
Contributor

@tejal29 tejal29 commented Jun 6, 2020

small perf improvement.

  1. check if a file is changed in gowalkdir which is faster than filepath.Walk.
  2. Move file deletion logic in the same loop.

This PR reduces the number of times the whole filesystem is walked from 3 to just 1.

@googlebot googlebot added the cla: yes CLA signed by all commit authors label Jun 6, 2020
@tejal29
Copy link
Copy Markdown
Contributor Author

tejal29 commented Jun 6, 2020

Performance improvements


Number of Files | Total Build Time Full Snapshot - current | Total Build Time -  Walk performance | % decrease
-- | -- | -- | --
10000 | 13.834973 | 13.579324 | 1.85%
50000 | 47.926338 | 42.460665 | 11.40%
100000 | 96.440554 | 77.774629 | 19.35%
200000 | 187.346273 | 175.198485 | 6.48%
300000 | 341.233622 | 289.063343 | 15.29%
500000 | 1404.410121 | 1122.947834 | 20.04%
700000 | 1772.379601 | 1350.19682 | 23.82%

@tejal29 tejal29 merged commit e73dea4 into GoogleContainerTools:master Jun 6, 2020
@tejal29 tejal29 added the area/performance issues related to kaniko performance enhancement label Aug 22, 2020
@tejal29 tejal29 deleted the check_file_changed_in_loop branch April 26, 2021 16:42
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

area/performance issues related to kaniko performance enhancement cla: yes CLA signed by all commit authors

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants