Add buffering for large layers.#428
Add buffering for large layers.#428dlorenc merged 1 commit intoGoogleContainerTools:masterfrom dlorenc:buffer
Conversation
| // TakeSnapshot takes a snapshot of the specified files, avoiding directories in the whitelist, and creates | ||
| // a tarball of the changed files. Return contents of the tarball, and whether or not any files were changed | ||
| func (s *Snapshotter) TakeSnapshot(files []string) ([]byte, error) { | ||
| buf := bytes.NewBuffer([]byte{}) |
There was a problem hiding this comment.
it looks like you decided to move this buffering up a layer - personally i think it's nicer for the caller not to have to worry about it but either way, maybe the commit message could go into some detail about why that choice was made?
| continue | ||
| } | ||
|
|
||
| f, err := ioutil.TempFile("/kaniko", "") |
There was a problem hiding this comment.
is there any case where we would want this to be in-memory instead? (i.e. configurable)
There was a problem hiding this comment.
I don't think so - I think if anything the path itself here would be configurable. Using a tmp or memfs directory should be basically the same thing as in memory vs. on disk.
dlorenc
left a comment
dlorenc
left a comment
There was a problem hiding this comment.
OK, cleaned this up quite a bit. Ready for another look.
priyawadhwa
left a comment
priyawadhwa
left a comment
There was a problem hiding this comment.
One nit, but otherwise LGTM!
When building Docker images, layers were previously stored in memory. This caused obvious issues when manipulating large layers, which could cause Kaniko to crash.
4 participants
This writes layer tarballs to disk rather than keeping them all in memory.
#358