-
Notifications
You must be signed in to change notification settings - Fork 347
Replacing containerd GRPC API with client #113
Conversation
70fd4fb to
4e5f5d1
Compare
|
@abhinandanpb Thanks! Ping me when you are ready, and feel free to tell me if you have any questions. :) |
3468950 to
0b3953c
Compare
542bed6 to
d51d23c
Compare
Random-Liu
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Haven't finished reviewing.
Send out some comments first.
| assert := assertlib.New(t) | ||
| sandboxes := map[string]Sandbox{} | ||
| for _, id := range ids { | ||
| sandboxes[id] = Sandbox{metadatas[id]} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sandbox{Metadata: metadatas[id]}
pkg/store/sandbox/sandbox.go
Outdated
| @@ -28,6 +28,8 @@ type Sandbox struct { | |||
| // Metadata is the metadata of the sandbox, it is immutable after created. | |||
| Metadata | |||
| // TODO(random-liu): Add containerd container client. | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Remove the TODO.
| Metadata | ||
| // TODO(random-liu): Add containerd container client. | ||
| // Containerd sandbox | ||
| Container containerd.Container |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Seems better to call this sandbox container.
// Containerd sandbox container
pkg/store/sandbox/sandbox.go
Outdated
|
|
||
| "github.com/containerd/containerd" | ||
| "github.com/kubernetes-incubator/cri-containerd/pkg/store" | ||
| "sync" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: Move "sync" up. Our style is usually:
import (
"sync" // golang libraries
"github.com/containerd/containerd" // Other libraries
"github.com/kubernetes-incubator/cri-containerd/pkg/store" // cri-containerd libraries
)
pkg/store/image/image.go
Outdated
| Config *imagespec.ImageConfig | ||
| // container image reference | ||
| Image containerd.Image | ||
| // TODO(random-liu): Add containerd image client. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Remove the TODO.
And // Containerd image reference seems better.
pkg/server/sandbox_run.go
Outdated
| containerd.WithNewSnapshotView(id, image.Image)}...) | ||
|
|
||
| var container containerd.Container | ||
| if container, err = c.client.NewContainer(ctx, id, opts...); err != nil { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit:
container, err := c.client.NewContainer(ctx, id, opts...)
if err != nil {
}
pkg/server/sandbox_run.go
Outdated
| containerd.WithSpec(spec), | ||
| containerd.WithImage(image.Image), | ||
| containerd.WithRuntime(defaultRuntime), | ||
| containerd.WithNewSnapshotView(id, image.Image)}...) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There is a problem with the containerd client. If NewContainer fails, no one will cleanup the snapshot.
In our original implementation, we'll remove the snapshot. But in their client, it's not removed.
This is not your problem, will file an issue to them. But could you add a TODO here to track that?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks like I removed the TODO i had added earlier. I can fix it in containerd
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Cool. Then let's add TODO on our side, and try to fix it in containerd. :)
pkg/server/sandbox_run.go
Outdated
| // Start sandbox container in containerd. | ||
| if _, err := c.taskService.Start(ctx, &tasks.StartTaskRequest{ContainerID: id}); err != nil { | ||
| return nil, fmt.Errorf("failed to start sandbox container %q: %v", | ||
| err = task.Start(ctx) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit to save one line:
if err := task.Start(ctx); err != nil {
}
pkg/server/sandbox_run.go
Outdated
| if retErr != nil { | ||
| // Cleanup the sandbox container if an error is returned. | ||
| if err := c.stopSandboxContainer(ctx, id); err != nil { | ||
| if _, err := task.Delete(ctx); err != nil { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You could not Delete a task once it's created, unless you use the new WithProcessKill option containerd/containerd#1301.
We need the orignal code here, and replace it with Delete WithProcessKill after we bump up containerd version.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For this PR to work we anyways need to bump containerd version. I can incorporate the change. WDYT ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure. Let's update containerd version then.
There may be some change in their api and client, you may need to update your PR correspondingly.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Delete with kill looks useful
containerd/containerd@d8c075a
pkg/server/sandbox_run.go
Outdated
| if err != nil { | ||
| return nil, fmt.Errorf("failed to create sandbox container %q: %v", | ||
| id, err) | ||
| return nil, fmt.Errorf("failed to create task for sandbox:%q: %v", container.ID(), err) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: s/sandbox:%q/sandbox %q.
896de47 to
35338e4
Compare
| import ( | ||
| "sync" | ||
|
|
||
| "github.com/containerd/containerd" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: Maybe a little bit too picky, but may help with future PRs, :p
import(
"sync" // golang libraries
// empty line
"github.com/containerd/containerd" // other libraries
// empty line
"github.com/kubernetes-incubator/cri-containerd/pkg/store" // CRI containerd libraries
)
And also the same with the other imports.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I use goimports to format. I will skip using that.
pkg/server/status.go
Outdated
| resp, err := c.healthService.Check(ctx, &healthapi.HealthCheckRequest{}) | ||
| if err != nil || resp.Status != healthapi.HealthCheckResponse_SERVING { | ||
|
|
||
| if c.client == nil { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Seems not required. Because we've been using c.client.XXX all around, :)
We've made the assumption that once grpc handler runs, client should be ready.
Random-Liu
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not finished yet
pkg/server/sandbox_run.go
Outdated
| return nil, fmt.Errorf("failed to start sandbox container task %q: %v", | ||
| id, err) | ||
| } | ||
| defer func() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is not required. We only need one defer after NewTask to Delete WithProcessKill.
pkg/server/sandbox_run.go
Outdated
|
|
||
| sandbox.Pid = createResp.Pid | ||
| sandbox.NetNS = getNetworkNamespace(createResp.Pid) | ||
| // Create sandbox task in containerd. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why move this down here?
pkg/server/sandbox_run.go
Outdated
| if err != nil { | ||
| return nil, fmt.Errorf("failed to create sandbox container %q: %v", | ||
| id, err) | ||
| return nil, fmt.Errorf("failed to create task for sandbox %q: %v", container.ID(), err) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
s/container.ID()/id.
id seems more straightforward as sandbox id.
pkg/server/sandbox_run.go
Outdated
| if err != nil { | ||
| return nil, fmt.Errorf("failed to create containerd container: %v", err) | ||
| } | ||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: unnecessary empty line.
pkg/server/sandbox_run.go
Outdated
| glog.V(5).Infof("Create sandbox container (id=%q, name=%q) with options %+v.", | ||
| id, name, createOpts) | ||
| createResp, err := c.taskService.Create(ctx, createOpts) | ||
| task, err := container.NewTask(ctx, containerd.NewIOWithFifoSet(os.Stdin, os.Stdout, os.Stderr, fifo, false)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's fine to use containerd.NewFifos and create fifos in /tmp. However, the problem is that there is no way to pass empty stdin now.
There are 3 options:
- Option 1: We create fifos ourselves, and pass them to containerd client. (What you are doing).
- Option 2: Still use
containerd.NewIO, but add option to disable stdin. - Option 3: Use
containerd.NewIOdirectly, and calltask.CloseIOafterwards for containers which don't want stdin.
I feel that option 2 or 3 seems better, because less change is required.
Will move this discussion to your containerd PR.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Another problem with PR. Why using os.Stdin, os.Stdout and os.Stderr.
Should be nil, ioutil.Discard and ioutil.Discard.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is still possible. The only reason I avoided using NewIO as is -> we need to pass the customer path. The tests were failing because CRI validation checks for logs in /tmp/Podxxxx where containerd by default (NewIO) logs into /tmp/containerd/XXXX.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@abhinandanpb That's not how it works.
Containerd create FIFOs for each container. Currently, we create those fifos and give the path to containerd. In fact, we could use containerd client directly, which creates fifos for us in /tmp/containerd/XXX. We don't really care where the FIFOs are.
We create a Logger for each container, which read log from the FIFO, decorate the log, and write to the CRI log file, which is /var/log/pods/XXX. The test is checking this path, it doesn't care where the FIFOs are.
NewIO accepts streams directly, e.g. for stdout, we just need to open a pipe, pass the writer side to NewIO, pass the reader side to logger.
os.Stdout/os.Stderr/os.Stdin should not be used here.
| if err != nil { | ||
| return nil, fmt.Errorf("failed to resolve image %q: %v", imageRef, err) | ||
| } | ||
| if image == nil { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For localResolve, we still need this.
pkg/server/container_create.go
Outdated
| }, | ||
| RootFS: id, | ||
| }); err != nil { | ||
| opts = append(opts, []containerd.NewContainerOpts{containerd.WithSpec(spec), containerd.WithRuntime(defaultRuntime)}...) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just:
opts = append(opts,
containerd.WithSpec(spec),
containerd.WithRuntime(defaultRuntime))
pkg/server/container_create.go
Outdated
| return nil, fmt.Errorf("failed to create internal container object for %q: %v", | ||
| id, err) | ||
| } | ||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why add a new line before defer? I feel like we should group defer with the resource we want it to release?
Or is there any other convention?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i usally let gofmt/goimports take care of formatting and indentation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@abhinandanpb Weird. In my case, gofmt never add unnecessary new lines for me.
pkg/server/container_create.go
Outdated
| }() | ||
|
|
||
| container, err := containerstore.NewContainer(meta, containerstore.Status{CreatedAt: time.Now().UnixNano()}) | ||
| container, err := containerstore.NewContainer(meta, containerstore.Status{CreatedAt: time.Now().UnixNano()}, newContainer) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: The line is too long, add a new line?
pkg/server/container_create.go
Outdated
| opts = append(opts, []containerd.NewContainerOpts{containerd.WithSpec(spec), containerd.WithRuntime(defaultRuntime)}...) | ||
|
|
||
| var newContainer containerd.Container | ||
| if newContainer, err = c.client.NewContainer(ctx, id, opts...); err != nil { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit:
newContainer, err := c.client.XXXXXXXX
if err != nil {
return nil, fmt.Errorf("failed to create containerd container: %v", err)
}newContainer/container seems a bit confusing...Let's use cntr or cContainer or something else here?
pkg/server/container_start.go
Outdated
| // startContainer actually starts the container. The function needs to be run in one transaction. Any updates | ||
| // to the status passed in will be applied no matter the function returns error or not. | ||
| func (c *criContainerdService) startContainer(ctx context.Context, id string, meta containerstore.Metadata, status *containerstore.Status) (retErr error) { | ||
| func (c *criContainerdService) startContainer(ctx context.Context, container containerd.Container, meta containerstore.Metadata, status *containerstore.Status) (retErr error) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: The line is too long, add a new line.
pkg/server/container_start.go
Outdated
| if _, err := c.taskService.Delete(ctx, &tasks.DeleteTaskRequest{ContainerID: id}); err != nil { | ||
| glog.Errorf("Failed to delete containerd task %q: %v", id, err) | ||
| if _, err := task.Delete(ctx); err != nil { | ||
| glog.Errorf("Failed to delete container %q: %v", container.ID(), err) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
original error message seems better.
pkg/server/container_start.go
Outdated
| // Cleanup the containerd task if an error is returned. | ||
| if _, err := c.taskService.Delete(ctx, &tasks.DeleteTaskRequest{ContainerID: id}); err != nil { | ||
| glog.Errorf("Failed to delete containerd task %q: %v", id, err) | ||
| if _, err := task.Delete(ctx); err != nil { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should add WithProcessKill.
My original code was not correct.
pkg/server/container_stop.go
Outdated
| } | ||
|
|
||
| err = task.Kill(ctx, stopSignal) | ||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
? unnecessary empty line? There are several empty lines, seem like something wrong with your editor?
if err := task.Kill(ctx, stopSignal); err != nil {
}
pkg/server/container_stop.go
Outdated
| Signal: uint32(unix.SIGKILL), | ||
| All: true, | ||
| }) | ||
| err = task.Kill(ctx, unix.SIGKILL) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if err := task.Kill(ctx, unix.SIGKILL); err != nil {
}
pkg/server/sandbox_run.go
Outdated
| glog.V(5).Infof("Create sandbox container (id=%q, name=%q) with options %+v.", | ||
| id, name, createOpts) | ||
| createResp, err := c.taskService.Create(ctx, createOpts) | ||
| task, err := container.NewTask(ctx, containerd.NewIOWithFifoSet(os.Stdin, os.Stdout, os.Stderr, fifo, false)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@abhinandanpb That's not how it works.
Containerd create FIFOs for each container. Currently, we create those fifos and give the path to containerd. In fact, we could use containerd client directly, which creates fifos for us in /tmp/containerd/XXX. We don't really care where the FIFOs are.
We create a Logger for each container, which read log from the FIFO, decorate the log, and write to the CRI log file, which is /var/log/pods/XXX. The test is checking this path, it doesn't care where the FIFOs are.
NewIO accepts streams directly, e.g. for stdout, we just need to open a pipe, pass the writer side to NewIO, pass the reader side to logger.
os.Stdout/os.Stderr/os.Stdin should not be used here.
Random-Liu
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
First pass review finished.
Will do the second pass after you reply and address comments.
| if err != nil { | ||
| if !isContainerdGRPCNotFoundError(err) { | ||
| return fmt.Errorf("failed to stop container, task not found for container %q: %v", id, err) | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
task will be nil if there is an error, we may want to skip the task.Kill when there is an error, even it's not found error.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
And we could get task outside of the if clause, so that we don't need to do it twice.
pkg/server/container_stop.go
Outdated
|
|
||
| task, err := container.Container.Task(ctx, nil) | ||
| if err != nil { | ||
| if !isContainerdGRPCNotFoundError(err) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Skip task.Kill if not found.
pkg/server/container_stop_test.go
Outdated
| container, err := containerstore.NewContainer( | ||
| containerstore.Metadata{ID: id}, | ||
| *test.status, | ||
| *test.status, nil, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
New line.
| return "", 0, nil, fmt.Errorf("failed to get image %q from containerd image store: %v", | ||
| normalizedRef, err) | ||
| } | ||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should add following functions into containerd client.
Image.Config, Image.RootFS, Image.Size etc.
| // 2) We need desc returned by schema1 converter. | ||
| // So just put the image metadata after downloading now. | ||
| // TODO(random-liu): Fix the potential garbage collection race. | ||
| repoDigest, repoTag := getRepoDigestAndTag(namedRef, desc.Digest, schema1Converter != nil) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We still need to write repoDigest, repoTag and imageID into containerd image metadata store.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you elaborate on this .? This has to be written on the containerd side ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Currently, our internal image store is maintaining the Image ID -> RepoDigests/RepoTags mapping, which is for the requirement of CRI.
However, when user create a container or pulling a image, they usually give us RepoDigest/RepoTag. So we have to do the RepoDigest/RepoTag -> Image ID (and other Information) conversion, which instead of creating another internal store, we are currently leveraging containerd image metadata store to do that. That's why we should put all RepoDigest/RepoTag into the metadata store.
As for why we also need to put image ID: I was told that containerd will do image gc based on reference, an entry in image metadata store is a reference to the corresponding snapshot and content. Other references are not unique enough to identify an image, e.g. repo tag could be reused by another image; schema 1 image doesn't have a repo digest etc.; image id seems to be the only thing we can use to reference the image and keep it from being garbage collected.
69a3d13 to
763742a
Compare
ca7eac9 to
0bbdbfe
Compare
| }() | ||
|
|
||
| container, err := containerstore.NewContainer(meta, containerstore.Status{CreatedAt: time.Now().UnixNano()}) | ||
| container, err := containerstore.NewContainer(meta, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: Can we follow this pattern?
That is the containerd/grpc pattern which seems more clean, and we don't need to update unit test everywhere.
pkg/server/container_start.go
Outdated
| return fmt.Errorf("sandbox container %q is not running", sandboxID) | ||
| taskStatus, err := s.Status(ctx) | ||
| if err != nil { | ||
| return fmt.Errorf("faile to get task status for container %q : %v", id, err) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
s/faile/failed
s/for container/for sandbox container.
pkg/server/container_start.go
Outdated
| glog.V(5).Infof("Create containerd task (id=%q, name=%q) with options %+v.", | ||
| id, meta.Name, createOpts) | ||
| createResp, err := c.taskService.Create(ctx, createOpts) | ||
| task, err := container.NewTask(ctx, containerd.NewIO(os.Stdin, wStdoutPipe, wStderrPipe)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
s/os.Stdin/new(bytes.Buffer).
And please add TODO to close container stdin if not needed.
| if !errdefs.IsNotFound(err) { | ||
| return fmt.Errorf("failed to stop container, task not found for container %q: %v", id, err) | ||
| } | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Use else?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actually it's a bit troublesome that we have to load the task again whenever we want to use it. :p
| if err != nil { | ||
| if !isContainerdGRPCNotFoundError(err) && !isRuncProcessAlreadyFinishedError(err) { | ||
| return fmt.Errorf("failed to kill container %q: %v", id, err) | ||
| if task != nil { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
else?
pkg/server/sandbox_stop.go
Outdated
| } | ||
| taskStatus, err := task.Status(ctx) | ||
| if err != nil { | ||
| return fmt.Errorf("failed to get task status for sandbox %q : %v", container.ID(), err) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Remove space before :.
Actually, now I prefer not to include id inside error message, unless the caller doesn't know that. Currently, error message is a bit messy. :p We need to gradually clean them up.
pkg/server/sandbox_run.go
Outdated
| if retErr != nil { | ||
| // Cleanup the sandbox container if an error is returned. | ||
| if err := c.stopSandboxContainer(ctx, id); err != nil { | ||
| if err := c.stopSandboxContainer(ctx, container); err != nil { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since you've got task here, just call task.Delete(ctx, WithProcessKill).
| func (c *criContainerdService) stopSandboxContainer(ctx context.Context, id string) error { | ||
| cancellable, cancel := context.WithCancel(ctx) | ||
| eventstream, err := c.eventService.Subscribe(cancellable, &events.SubscribeRequest{}) | ||
| func (c *criContainerdService) stopSandboxContainer(ctx context.Context, container containerd.Container) error { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This function is not that useful now, maybe just embed into StopContainer.
pkg/server/sandbox_status.go
Outdated
| if task != nil { | ||
| taskStatus, err := task.Status(ctx) | ||
| if err != nil { | ||
| return nil, fmt.Errorf("failed to get task status for sandbox container %q : %v", id, err) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: space before :.
| } | ||
| glog.V(4).Infof("Pulled image %q with image id %q, repo tag %q, repo digest %q", imageRef, imageID, | ||
| repoDigest, repoTag := getRepoDigestAndTag(namedRef, image.Target().Digest, image.Target().MediaType == containerdimages.MediaTypeDockerSchema1Manifest) | ||
| for _, r := range []string{repoTag, repoDigest} { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We also need to put image ID. :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thats already taken care of by the client ? https://github.com/containerd/containerd/blob/master/client.go#L252
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@abhinandanpb The client only puts the image reference in.
There are several different image references:
- Image ID: Image config digest.
- Image Pullable Digest: Image manifest digest.
- Image ChainID: Calculated from uncompressed image snapshot.
- Image Name/Tag: Readable image name, e.g docker.io/library/busybox:latest, docker.io/library/nginx:latest etc.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This issue kubernetes/kubernetes#46255, and the OCI image spec document may be useful here. :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Got it . Thanks for explaining. Is there anyway we could avoid storing these many-to-one reference copies in the store ? I mean any change we can do on the containerd side to resolve the image ? Not in the scope of this PR. But moving forward.
pkg/server/image_pull.go
Outdated
| repoTag, repoDigest) | ||
|
|
||
| // Get image information. | ||
| chainID, size, config, err := c.getImageInfo(ctx, imageRef) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
getImageInfo may not be needed, if Image.Size(), Image.RootFS() could provide enough information.
054746b to
73ec2fe
Compare
Random-Liu
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Several final nits. LGTM overall.
pkg/server/container_start.go
Outdated
| return fmt.Errorf("sandbox container %q is not running", sandboxID) | ||
| taskStatus, err := s.Status(ctx) | ||
| if err != nil { | ||
| return fmt.Errorf("failed to get task status for sandbox container %q : %v", id, err) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: remove space before :.
pkg/server/image_pull.go
Outdated
| return nil, fmt.Errorf("failed to update image reference %q: %v", r, err) | ||
| } | ||
| } | ||
| glog.V(4).Infof("Pulled image %q with image id %q, repo tag %q, repo digest %q", imageRef, image.Name(), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Log this after you get the image id, so that you could log the image id out.
pkg/server/helpers.go
Outdated
| @@ -272,43 +244,45 @@ func normalizeImageRef(ref string) (reference.Named, error) { | |||
| // getImageInfo returns image chainID, compressed size and oci config. Note that getImageInfo | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
and image id
| @@ -272,43 +244,45 @@ func normalizeImageRef(ref string) (reference.Named, error) { | |||
| // getImageInfo returns image chainID, compressed size and oci config. Note that getImageInfo | |||
| // assumes that the image has been pulled or it will return an error. | |||
| func (c *criContainerdService) getImageInfo(ctx context.Context, ref string) ( | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: Return image id as imagedigest.Digest.
pkg/server/image_pull.go
Outdated
| image := imagestore.Image{ | ||
| imageID := desc.Digest.String() | ||
| if err := c.createImageReference(ctx, imageID, image.Target()); err != nil { | ||
| return nil, fmt.Errorf("failed to update image reference %q: %v", r, err) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why r here?
Actually, we could get image id before updating repoDigest, repoTag, so that we could update them all together in a single loop.
|
@Random-Liu updated PR. Thanks for reviewing the changes multiple times :) |
|
@abhinandanpb LGTM, let's wait for the test result. |
pkg/server/helpers.go
Outdated
| } | ||
|
|
||
| // getImageInfo returns image chainID, compressed size and oci config. Note that getImageInfo | ||
| // getImageInfo returns image chainID, compressed size, imageID and oci config. Note that getImageInfo |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
oci config and imageID.
To make it the same with the return order.
pkg/server/image_pull.go
Outdated
| return nil, fmt.Errorf("failed to get image %q information: %v", imageRef, err) | ||
| } | ||
| imageID := id.String() | ||
| glog.V(4).Infof("Pulled image %q with image id %q, repo tag %q, repo digest %q", imageRef, imageID, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please move this down after the following if clause.
This commit: 1) Replaces the usage of containerd GRPC APIs with the containerd client for all operations related to containerd. 2) Updated containerd to v1.0alpha4+ 3) Updated runc to v1.0.0 Signed-off-by: Abhinandan Prativadi <[email protected]>
|
Done. |
|
LGTM |
Replacing containerd GRPC API with client
This commit
Signed-off-by: Abhinandan Prativadi [email protected]