Skip to content
This repository was archived by the owner on Jun 3, 2025. It is now read-only.

Support multi stage builds#141

Merged
priyawadhwa merged 12 commits intoGoogleContainerTools:masterfrom
priyawadhwa:multistage
May 14, 2018
Merged

Support multi stage builds#141
priyawadhwa merged 12 commits intoGoogleContainerTools:masterfrom
priyawadhwa:multistage

Conversation

@priyawadhwa
Copy link
Copy Markdown
Contributor

Fixes #4 and partially #9

Copy link
Copy Markdown
Contributor

@dlorenc dlorenc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense overall, a few high level comments. I'll take a closer look once we rebase this on top of the go-containers switch.

Comment thread pkg/commands/copy.go
logrus.Infof("dest: %s", dest)

// Resolve from
if c.cmd.From != "" {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if multiple steps have the same from line? Should we add any randomness here?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, what do you mean by randomness?

Right now, if "from" is specified as a previous stage, we change the build context to be the path to the directory where files from that stage are saved. That directory should remain unchanged even if multiple copy commands refer back to it

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was worried about a case like this:

FROM base1
RUN something

FROM base1
RUN somethingelse

FROM base2
COPY --from=0

But I guess we can't have conflicts here, because the users have to refer to them by index (or alias)

Comment thread pkg/dockerfile/dockerfile.go Outdated
nameToIndex := make(map[string]string)
for i, stage := range stages {
index := strconv.Itoa(i)
nameToIndex[stage.Name] = index
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it looks like the second one here overwrites the first one. Do we need both?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so if the stage has a name associated the second won't override the first, so I added both in just to make it easier to resolve '--from' from names to indexes later on

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah ok. Maybe it would be more clear to have two separate maps?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess it would get confusing if someone tried to name a stage with an index...

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh I see, that makes sense -- I changed it a bit to be less confusing with one map

Comment thread pkg/dockerfile/dockerfile.go Outdated
if stageIndex <= index {
continue
}
ms, err := image.NewSourceImage(stage.BaseName)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will have to get rewritten a bit after my go-containerregistry PR goes in.

@priyawadhwa
Copy link
Copy Markdown
Contributor Author

This is RFAL!

Comment thread cmd/executor/cmd/root.go Outdated
srcContext string
snapshotMode string
bucket string
logLevel string
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks like you dropped the dockerinsecuretlsverify in a merge conflict.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yah, it didn't look like it was doing anything now that we switched to go-containerregistry

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(probably should have done that in a different commit)

Comment thread pkg/commands/copy.go
logrus.Infof("dest: %s", dest)

// Resolve from
if c.cmd.From != "" {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was worried about a case like this:

FROM base1
RUN something

FROM base1
RUN somethingelse

FROM base2
COPY --from=0

But I guess we can't have conflicts here, because the users have to refer to them by index (or alias)

Comment thread pkg/dockerfile/dockerfile.go Outdated
nameToIndex := make(map[string]string)
for i, stage := range stages {
index := strconv.Itoa(i)
nameToIndex[stage.Name] = index
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah ok. Maybe it would be more clear to have two separate maps?

Comment thread pkg/dockerfile/dockerfile.go Outdated
nameToIndex := make(map[string]string)
for i, stage := range stages {
index := strconv.Itoa(i)
nameToIndex[stage.Name] = index
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess it would get confusing if someone tried to name a stage with an index...

Comment thread pkg/dockerfile/dockerfile.go Outdated
switch c := cmd.(type) {
case *instructions.EnvCommand:
envCommand := commands.NewEnvCommand(c)
if err := envCommand.ExecuteCommand(&imageConfig.Config); err != nil {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are we executing ENV commands in here? It seems like this shouldn't have any side effects.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so we need to keep track of which ENVs were declared before a copy command for a case like this:

FROM base1
RUN something

FROM base2
ENV file file
COPY --from=0 $file ./

So here I'm executing any EnvCommands on a temporary config for that stage, and that updated config is used to evaluate dependencies in any subsequent COPY commands

@priyawadhwa
Copy link
Copy Markdown
Contributor Author

PTAL

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

area/multi-stage builds issues related to kaniko multi-stage builds

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants