Skip to content
This repository was archived by the owner on Jun 3, 2025. It is now read-only.

feat: optimize build#694

Merged
tejal29 merged 4 commits intoGoogleContainerTools:masterfrom
orisano:feat-optimize-build
Oct 4, 2019
Merged

feat: optimize build#694
tejal29 merged 4 commits intoGoogleContainerTools:masterfrom
orisano:feat-optimize-build

Conversation

@orisano
Copy link
Copy Markdown
Contributor

@orisano orisano commented Jun 4, 2019 •
edited
Loading

  • replace md5 to highwayhash
  • optimize HasFilepathPrefix

Result in my project:
runtime.main: 87.51s -> 55.40s
total: 129.54s -> 88.29s

before:

profile001

after:

profile002

@orisano orisano force-pushed the feat-optimize-build branch from c798a46 to 4647b3e Compare June 4, 2019 18:31
@orisano
Copy link
Copy Markdown
Contributor Author

orisano commented Jun 4, 2019

Bench Result of HasFilepathPrefix.

name                                           old time/op    new time/op    delta
HasFilepathPrefix/PathDepth=2,PrefixDepth=1-4     297ns ± 9%     289ns ±21%     ~     (p=0.726 n=10+10)
HasFilepathPrefix/PathDepth=3,PrefixDepth=1-4     298ns ± 3%     270ns ± 6%   -9.16%  (p=0.000 n=10+9)
HasFilepathPrefix/PathDepth=4,PrefixDepth=1-4     332ns ± 4%     289ns ±13%  -13.04%  (p=0.000 n=10+10)
HasFilepathPrefix/PathDepth=5,PrefixDepth=1-4     384ns ±14%     319ns ± 7%  -16.92%  (p=0.000 n=10+10)
HasFilepathPrefix/PathDepth=2,PrefixDepth=2-4     303ns ±14%     337ns ±38%     ~     (p=0.133 n=10+9)
HasFilepathPrefix/PathDepth=3,PrefixDepth=2-4     362ns ±17%     346ns ±13%     ~     (p=0.247 n=10+10)
HasFilepathPrefix/PathDepth=4,PrefixDepth=2-4     427ns ±22%     390ns ±16%     ~     (p=0.086 n=10+10)
HasFilepathPrefix/PathDepth=5,PrefixDepth=2-4     403ns ± 9%     347ns ± 6%  -13.88%  (p=0.000 n=9+9)
HasFilepathPrefix/PathDepth=2,PrefixDepth=3-4     342ns ± 7%     353ns ±13%     ~     (p=0.530 n=9+9)
HasFilepathPrefix/PathDepth=3,PrefixDepth=3-4     413ns ±15%     353ns ± 8%  -14.39%  (p=0.001 n=10+10)
HasFilepathPrefix/PathDepth=4,PrefixDepth=3-4     531ns ±28%     383ns ± 3%  -27.77%  (p=0.000 n=9+10)
HasFilepathPrefix/PathDepth=5,PrefixDepth=3-4     485ns ±29%     424ns ±19%  -12.53%  (p=0.009 n=10+10)

name                                           old alloc/op   new alloc/op   delta
HasFilepathPrefix/PathDepth=2,PrefixDepth=1-4     80.0B ± 0%     80.0B ± 0%     ~     (all equal)
HasFilepathPrefix/PathDepth=3,PrefixDepth=1-4     96.0B ± 0%     80.0B ± 0%  -16.67%  (p=0.000 n=10+10)
HasFilepathPrefix/PathDepth=4,PrefixDepth=1-4      112B ± 0%       80B ± 0%  -28.57%  (p=0.000 n=10+10)
HasFilepathPrefix/PathDepth=5,PrefixDepth=1-4      128B ± 0%       80B ± 0%  -37.50%  (p=0.000 n=10+10)
HasFilepathPrefix/PathDepth=2,PrefixDepth=2-4     96.0B ± 0%    112.0B ± 0%  +16.67%  (p=0.000 n=10+10)
HasFilepathPrefix/PathDepth=3,PrefixDepth=2-4      112B ± 0%      112B ± 0%     ~     (all equal)
HasFilepathPrefix/PathDepth=4,PrefixDepth=2-4      128B ± 0%      112B ± 0%  -12.50%  (p=0.000 n=10+10)
HasFilepathPrefix/PathDepth=5,PrefixDepth=2-4      144B ± 0%      112B ± 0%  -22.22%  (p=0.000 n=10+10)
HasFilepathPrefix/PathDepth=2,PrefixDepth=3-4      112B ± 0%      144B ± 0%  +28.57%  (p=0.000 n=10+10)
HasFilepathPrefix/PathDepth=3,PrefixDepth=3-4      128B ± 0%      144B ± 0%  +12.50%  (p=0.000 n=10+10)
HasFilepathPrefix/PathDepth=4,PrefixDepth=3-4      144B ± 0%      144B ± 0%     ~     (all equal)
HasFilepathPrefix/PathDepth=5,PrefixDepth=3-4      160B ± 0%      144B ± 0%  -10.00%  (p=0.000 n=10+10)

name                                           old allocs/op  new allocs/op  delta
HasFilepathPrefix/PathDepth=2,PrefixDepth=1-4      2.00 ± 0%      2.00 ± 0%     ~     (all equal)
HasFilepathPrefix/PathDepth=3,PrefixDepth=1-4      2.00 ± 0%      2.00 ± 0%     ~     (all equal)
HasFilepathPrefix/PathDepth=4,PrefixDepth=1-4      2.00 ± 0%      2.00 ± 0%     ~     (all equal)
HasFilepathPrefix/PathDepth=5,PrefixDepth=1-4      2.00 ± 0%      2.00 ± 0%     ~     (all equal)
HasFilepathPrefix/PathDepth=2,PrefixDepth=2-4      2.00 ± 0%      2.00 ± 0%     ~     (all equal)
HasFilepathPrefix/PathDepth=3,PrefixDepth=2-4      2.00 ± 0%      2.00 ± 0%     ~     (all equal)
HasFilepathPrefix/PathDepth=4,PrefixDepth=2-4      2.00 ± 0%      2.00 ± 0%     ~     (all equal)
HasFilepathPrefix/PathDepth=5,PrefixDepth=2-4      2.00 ± 0%      2.00 ± 0%     ~     (all equal)
HasFilepathPrefix/PathDepth=2,PrefixDepth=3-4      2.00 ± 0%      2.00 ± 0%     ~     (all equal)
HasFilepathPrefix/PathDepth=3,PrefixDepth=3-4      2.00 ± 0%      2.00 ± 0%     ~     (all equal)
HasFilepathPrefix/PathDepth=4,PrefixDepth=3-4      2.00 ± 0%      2.00 ± 0%     ~     (all equal)
HasFilepathPrefix/PathDepth=5,PrefixDepth=3-4      2.00 ± 0%      2.00 ± 0%     ~     (all equal)

@orisano orisano force-pushed the feat-optimize-build branch from 4647b3e to ace76f4 Compare June 4, 2019 18:35
@orisano
Copy link
Copy Markdown
Contributor Author

orisano commented Jul 28, 2019

ping

tejal29
tejal29 suggested changes Sep 17, 2019
View reviewed changes
Comment thread pkg/util/util.go
key := make([]byte, highwayhash.Size)
hasher := func(p string) (string, error) {
h := md5.New()
h, _ := highwayhash.New(key)
Copy link
Copy Markdown
Contributor

@tejal29 tejal29 Sep 17, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Check error. Can you explain or add link to benefits os highwayhash instead of md5.
Any disadvantages? Is this project being actively worked on?

Comment thread pkg/util/util.go
@tejal29
Copy link
Copy Markdown
Contributor

tejal29 commented Sep 17, 2019

@orisano This looks awesome! Thank you for all tour work!
Left a few comments.

@tejal29
Copy link
Copy Markdown
Contributor

tejal29 commented Sep 17, 2019

Looks like this failed

Building images for Dockerfile Dockerfile_test_meta_arg
Error building images: Failed to build image gcr.io/kaniko-test/docker-dockerfile_test_registry with kaniko command "[docker run -v /home/kbuilder/.config/gcloud:/root/.config/gcloud -v /tmpfs/tmp/622634064:/kaniko/benchmarks -v /usr/local/go/src/github.com/GoogleContainerTools/kaniko/integration:/workspace -e BENCHMARK_FILE=false executor-image -f /workspace/dockerfiles/Dockerfile_test_registry -d gcr.io/kaniko-test/kaniko-dockerfile_test_registry  -c /workspace]": exit status 1

@orisano orisano force-pushed the feat-optimize-build branch from a93b5a3 to 38fa360 Compare September 18, 2019 08:19
tejal29
tejal29 reviewed Sep 18, 2019
View reviewed changes
Comment thread pkg/util/util.go
if _, err := io.Copy(h, f); err != nil {
buf := pool.Get().(*[]byte)
defer pool.Put(buf)
if _, err := io.CopyBuffer(h, f, *buf); err != nil {
Copy link
Copy Markdown
Contributor

@tejal29 tejal29 Sep 18, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cool!

@tejal29 tejal29 merged commit 865d49c into GoogleContainerTools:master Oct 4, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

1 more reviewer

@tejal29 tejal29 tejal29 approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@orisano @tejal29 @kokoro-team