Skip to content

Dependent resources standalone mode #914

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 28 commits into from
Feb 11, 2022
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
5628db5
feat: refactor to support standalon mode
csviri Feb 7, 2022
0d2363c
fix: wip
csviri Feb 8, 2022
6dff17b
fix: wip
csviri Feb 8, 2022
ba619e5
fix: 1 IT
csviri Feb 8, 2022
dfa2388
fix: format
csviri Feb 8, 2022
1ad4a12
fix: IT wip
csviri Feb 9, 2022
4b089c1
fix: wip
csviri Feb 9, 2022
b3edc69
fix: IT test standalone resource
csviri Feb 9, 2022
7b73814
fix: refactor mysql sample
csviri Feb 9, 2022
76a6dc6
fix: wip
csviri Feb 9, 2022
04f76f9
Merge branch 'next' into dependent-resources-standalon-mode
csviri Feb 9, 2022
2d3289b
fix: merged next
csviri Feb 9, 2022
07b4834
Merge branch 'next' into dependent-resources-standalon-mode
csviri Feb 10, 2022
850a2ff
fix: tomcat e2e tests
csviri Feb 10, 2022
68a2619
fix: format
csviri Feb 10, 2022
73c76dc
fix: wip
csviri Feb 10, 2022
64b7148
fix: secret handling
csviri Feb 10, 2022
a3c7119
fix: formaty
csviri Feb 10, 2022
c22c295
fix: web page migrated to dependent resources
csviri Feb 10, 2022
3251067
fix: comment to revisit informer setting
csviri Feb 10, 2022
a3fb40f
fix: addressed issues from PR
csviri Feb 10, 2022
7843bbc
fix: comments from PR
csviri Feb 10, 2022
d851321
refactor: remove unneeded code
metacosm Feb 10, 2022
816757c
refactor: minor
metacosm Feb 10, 2022
5e25afa
refactor: better type safety, avoid duplicating configuration
metacosm Feb 10, 2022
da62a12
fix: null check
csviri Feb 11, 2022
1dc2d88
Merge branch 'dependent-resources-standalon-mode' of github.com:java-…
csviri Feb 11, 2022
7840238
fix: naming
csviri Feb 11, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
package io.javaoperatorsdk.operator.api.reconciler.dependent;

import io.fabric8.kubernetes.api.model.HasMetadata;
import io.javaoperatorsdk.operator.api.reconciler.Context;

public abstract class AbstractDependentResource<R, P extends HasMetadata>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it stays as a Factory (i.e. not exist = create) it should be renamed. And sharing version could be added (i.e. not exist = update status with missing target condition). The sharing version is another beast because, delete is about undoing what reconcile does not deleting. Again thinks of https://github.com/redhat-developer/service-binding-operator. When the CR is removed, the deployment is updated to suppress any env var that has been injected.
I don't if those two behaviors can be merge in one class

implements DependentResource<R, P> {

@Override
public void reconcile(P primary, Context context) {
var actual = getResource(primary);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

with Reconciler<T>, I guess the practice is to get the actual frrom context::getSecondaryResource. Shouldn't getResource have access to context ?
I like the way the caller of prepareEventSource actually take ownership and we don't need to keep a reference to it. It make no doubt on who should control its lifecycle (start/stop).

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, it's other way around, to for now the event sources are caching the resource ( this is an interesting thing we might want to re-evaluate this later, to separate these two ). But basically at the end the dependent resource should manage it itself, either having an event source (or directly calling the target API). So IMHO it should not have the context in getResource() basically forcing this pattern. (See KubernetesDependentResource)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would suggest to rename getResource to be more after the version of the resource it should get like actual/desired variant.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On level above on DependentResource we don't have desired, so it's really about reading the resource and the desired here is just a way to generically implement it. Hmm will think about it, so yes, it's always actual, not sure if needs to be that explicit.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@metacosm what do you think?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actual could make sense if we had desired at the same level, indeed. Maybe something that implies that the method is responsible for retrieving the resource more strongly? The previous implementation used getFor for this purpose but it's not really better. How about fetchFrom(primary) which would convey that the responsibility is to fetch the secondary based on the information provided by the primary?

var desired = desired(primary, context);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should avoid creating the desired version if it's not needed because it could be costly.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As far I can see it is always used on both branches.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this implementation, yes, that wasn't the case before :)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would still like to see this addressed if possible… In a subsequent PR, though.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can take a look together, not sure how you mean it.

if (actual.isEmpty()) {
create(desired, context);
} else {
if (!match(actual.get(), desired, context)) {
update(actual.get(), desired, context);
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if match doesn't assert that it has been reconciled. The secondary R may exist prior to the primary one. We should be able to put in the context what has been observed to be desired, so that the reconciler can update the status if it is empty.

In the case of mysql schema operator, if the database exists when the primary is created on the cluster, the status is never updated. But, it tricky here because the user is not symetric due to password.

if we split, schema and user in two different dependent. Thus, User::match cannot be sure actual = desired because the password may not be the desired one. So if the user exist prior to the CR, either it should fail the reconcile or the password should be changed and secret updated/recreated.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if match doesn't assert that it has been reconciled. The secondary R may exist prior to the primary one. We should be able to put in the context what has been observed to be desired, so that the reconciler can update the status if it is empty.

Not sure I'm following this. We read the R, and compare it to a target state, this is how P gets reconciled (at least this is a part of it).

In MySQL there is no update now available for the password. But the whole mysql we will revisit shortly.

}
}

protected abstract R desired(P primary, Context context);

protected boolean match(R actual, R target, Context context) {
return actual.equals(target);
}

protected abstract R create(R target, Context context);

// the actual needed to copy/preserve new labels or annotations
protected abstract R update(R actual, R target, Context context);

}
Original file line number Diff line number Diff line change
Expand Up @@ -9,54 +9,18 @@
import io.javaoperatorsdk.operator.processing.event.source.EventSource;

public interface DependentResource<R, P extends HasMetadata> {
default EventSource initEventSource(EventSourceContext<P> context) {
throw new IllegalStateException("Must be implemented if not automatically provided by the SDK");
}

Optional<EventSource> eventSource(EventSourceContext<P> context);

@SuppressWarnings("unchecked")
default Class<R> resourceType() {
return (Class<R>) Utils.getFirstTypeArgumentFromInterface(getClass());
}

default void delete(R fetched, P primary, Context context) {}

/**
* Computes the desired state of the dependent based on the state provided by the specified
* primary resource.
*
* The default implementation returns {@code empty} which corresponds to the case where the
* associated dependent should never be created by the associated reconciler or that the global
* state of the cluster doesn't allow for the resource to be created at this point.
*
* @param primary the primary resource associated with the reconciliation process
* @param context the {@link Context} associated with the reconciliation process
* @return an instance of the dependent resource matching the desired state specified by the
* primary resource or {@code empty} if the dependent shouldn't be created at this point
* (or ever)
*/
default Optional<R> desired(P primary, Context context) {
return Optional.empty();
}
void reconcile(P primary, Context context);

void delete(P primary, Context context);

Optional<R> getResource(P primaryResource);

/**
* Checks whether the actual resource as fetched from the cluster matches the desired state
* expressed by the specified primary resource.
*
* The default implementation always return {@code true}, which corresponds to the behavior where
* the dependent never needs to be updated after it's been created.
*
* Note that failure to properly implement this method will lead to infinite loops. In particular,
* for typical Kubernetes resource implementations, simply calling
* {@code desired(primary, context).equals(actual)} is not enough because metadata will usually be
* different.
*
* @param actual the current state of the resource as fetched from the cluster
* @param primary the primary resource associated with the reconciliation request
* @param context the {@link Context} associated with the reconciliation request
* @return {@code true} if the actual state of the resource matches the desired state expressed by
* the specified primary resource, {@code false} otherwise
*/
default boolean match(R actual, P primary, Context context) {
return true;
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
package io.javaoperatorsdk.operator.api.reconciler.dependent;

import java.util.Optional;

import io.fabric8.kubernetes.api.model.HasMetadata;
import io.fabric8.kubernetes.client.KubernetesClient;
import io.javaoperatorsdk.operator.api.config.informer.InformerConfiguration;
import io.javaoperatorsdk.operator.api.reconciler.Context;
import io.javaoperatorsdk.operator.api.reconciler.EventSourceContext;
import io.javaoperatorsdk.operator.processing.event.ResourceID;
import io.javaoperatorsdk.operator.processing.event.source.AssociatedSecondaryResourceIdentifier;
import io.javaoperatorsdk.operator.processing.event.source.EventSource;
import io.javaoperatorsdk.operator.processing.event.source.PrimaryResourcesRetriever;
import io.javaoperatorsdk.operator.processing.event.source.informer.InformerEventSource;
import io.javaoperatorsdk.operator.processing.event.source.informer.Mappers;

public abstract class KubernetesDependentResource<R extends HasMetadata, P extends HasMetadata>
extends AbstractDependentResource<R, P> {

private KubernetesClient client;
private boolean manageDelete;
private InformerEventSource<R, P> informerEventSource;

public KubernetesDependentResource() {
this(null, false);
}

public KubernetesDependentResource(KubernetesClient client) {
this(client, false);
}

public KubernetesDependentResource(KubernetesClient client, boolean manageDelete) {
this.client = client;
this.manageDelete = manageDelete;
}

@Override
protected R create(R target, Context context) {
return client.resource(target).createOrReplace();
}

@Override
protected R update(R actual, R target, Context context) {
// todo map annotation and labels ?
return client.resource(target).createOrReplace();
}

@Override
public Optional<EventSource> eventSource(EventSourceContext<P> context) {
if (informerEventSource != null) {
return Optional.of(informerEventSource);
}
var informerConfig = initInformerConfiguration(context);
informerEventSource = new InformerEventSource(informerConfig, context);
return Optional.of(informerEventSource);
}

private InformerConfiguration<R, P> initInformerConfiguration(EventSourceContext<P> context) {
PrimaryResourcesRetriever<R> associatedPrimaries =
(this instanceof PrimaryResourcesRetriever) ? (PrimaryResourcesRetriever<R>) this
: Mappers.fromOwnerReference();

AssociatedSecondaryResourceIdentifier<R> associatedSecondary =
(this instanceof AssociatedSecondaryResourceIdentifier)
? (AssociatedSecondaryResourceIdentifier<R>) this
: (r) -> ResourceID.fromResource(r);

return InformerConfiguration.from(context, resourceType())
.withPrimaryResourcesRetriever(associatedPrimaries)
.withAssociatedSecondaryResourceIdentifier(
(AssociatedSecondaryResourceIdentifier<P>) associatedSecondary)
.build();
}

public KubernetesDependentResource<R, P> withInformerEventSource(
InformerEventSource<R, P> informerEventSource) {
this.informerEventSource = informerEventSource;
return this;
}

@Override
public void delete(P primary, Context context) {
if (manageDelete) {
var resource = getResource(primary);
resource.ifPresent(r -> client.resource(r).delete());
}
}

@Override
public Optional<R> getResource(P primaryResource) {
return informerEventSource.getAssociated(primaryResource);
}

public KubernetesDependentResource<R, P> setClient(KubernetesClient client) {
this.client = client;
return this;
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
package io.javaoperatorsdk.operator.api.reconciler.dependent;

import java.util.Optional;

import io.fabric8.kubernetes.api.model.HasMetadata;
import io.javaoperatorsdk.operator.api.reconciler.EventSourceContext;
import io.javaoperatorsdk.operator.processing.event.source.EventSource;

// todo resource: resource which we don't manage just aware of, and needs it for input
//
public interface ObservedResource<R, P extends HasMetadata> {

default Optional<EventSource> initEventSource(EventSourceContext<P> context) {
return Optional.empty();
}

Optional<R> getResource();

}

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -8,28 +8,22 @@
import io.fabric8.kubernetes.api.model.HasMetadata;
import io.javaoperatorsdk.operator.api.config.dependent.DependentResourceConfiguration;
import io.javaoperatorsdk.operator.api.reconciler.Context;
import io.javaoperatorsdk.operator.api.reconciler.DeleteControl;
import io.javaoperatorsdk.operator.api.reconciler.EventSourceContext;
import io.javaoperatorsdk.operator.api.reconciler.Ignore;
import io.javaoperatorsdk.operator.api.reconciler.Reconciler;
import io.javaoperatorsdk.operator.api.reconciler.UpdateControl;
import io.javaoperatorsdk.operator.api.reconciler.dependent.DependentResource;
import io.javaoperatorsdk.operator.api.reconciler.dependent.Persister;
import io.javaoperatorsdk.operator.processing.event.source.EventSource;

@Ignore
public class DependentResourceController<R, P extends HasMetadata, C extends DependentResourceConfiguration<R, P>>
implements DependentResource<R, P>, Persister<R, P>, Reconciler<P> {
implements DependentResource<R, P> {

private static final Logger log = LoggerFactory.getLogger(DependentResourceController.class);

private final Persister<R, P> persister;
private final DependentResource<R, P> delegate;
protected final DependentResource<R, P> delegate;
private final C configuration;

public DependentResourceController(DependentResource<R, P> delegate, C configuration) {
this.delegate = delegate;
persister = initPersister(delegate);
this.configuration = configuration;
}

Expand All @@ -39,96 +33,30 @@ public Class<R> resourceType() {
}

@Override
public boolean match(R actual, P primary, Context context) {
return delegate.match(actual, primary, context);
public void delete(P primary, Context context) {
delegate.delete(primary, context);
}

@Override
public Optional<R> desired(P primary, Context context) {
return delegate.desired(primary, context);
public Optional<R> getResource(P primaryResource) {
return delegate.getResource(primaryResource);
}

@Override
public void delete(R fetched, P primary, Context context) {
delegate.delete(fetched, primary, context);
}

@SuppressWarnings("unchecked")
protected Persister<R, P> initPersister(DependentResource<R, P> delegate) {
if (delegate instanceof Persister) {
return (Persister<R, P>) delegate;
} else {
throw new IllegalArgumentException(
"DependentResource '" + delegate.getClass().getName() + "' must implement Persister");
}
}

public String descriptionFor(R resource) {
return resource.toString();
}

public Class<R> getResourceType() {
return delegate.resourceType();
}

@Override
public EventSource initEventSource(EventSourceContext<P> context) {
return delegate.initEventSource(context);
}

@Override
public void createOrReplace(R dependentResource, Context context) {
persister.createOrReplace(dependentResource, context);
public Optional<EventSource> eventSource(EventSourceContext<P> context) {
return delegate.eventSource(context);
}

@Override
public R getFor(P primary, Context context) {
return persister.getFor(primary, context);
}

public C getConfiguration() {
return configuration;
}

@Override
public UpdateControl<P> reconcile(P resource, Context context) {
var actual = getFor(resource, context);
if (actual == null || !match(actual, resource, context)) {
final var desired = desired(resource, context);
desired.ifPresent(d -> createOrReplaceDependent(resource, d, context));
}
return UpdateControl.noUpdate();
}

@Override
public DeleteControl cleanup(P primary, Context context) {
var dependent = getFor(primary, context);
if (dependent != null) {
delete(dependent, primary, context);
logOperationInfo(primary, dependent, "Deleting");
} else {
log.info("Ignoring already deleted {} for '{}' {}",
getResourceType().getName(),
primary.getMetadata().getName(),
primary.getKind());
}
return Reconciler.super.cleanup(primary, context);
public void reconcile(P resource, Context context) {
delegate.reconcile(resource, context);
}

protected void createOrReplaceDependent(P primary, R dependent, Context context) {
logOperationInfo(primary, dependent, "Reconciling");

// commit the changes
// todo: add metrics timing for dependent resource
createOrReplace(dependent, context);
}

private void logOperationInfo(P resource, R dependentResource, String operationDescription) {
if (log.isInfoEnabled()) {
log.info("{} {} for '{}' {}", operationDescription,
descriptionFor(dependentResource),
resource.getMetadata().getName(),
resource.getKind());
}
}
}
Loading