From 4fdf4730c502261663a34f7e520fea1ce1e0ff7e Mon Sep 17 00:00:00 2001 From: csviri Date: Wed, 15 Dec 2021 17:00:34 +0100 Subject: [PATCH 1/3] docs: skeleton --- docs/_data/sidebar.yml | 2 ++ .../architecture-and-internals.md | 18 ++++++++++++++++++ 2 files changed, 20 insertions(+) create mode 100644 docs/documentation/architecture-and-internals.md diff --git a/docs/_data/sidebar.yml b/docs/_data/sidebar.yml index 4963e97be1..5bd1f2ce22 100644 --- a/docs/_data/sidebar.yml +++ b/docs/_data/sidebar.yml @@ -11,6 +11,8 @@ url: /docs/patterns-best-practices - title: FAQ url: /docs/faq + - title: Architecture and Internals + url: /docs/architecture-and-internals - title: Contributing url: /docs/contributing - title: Migrating from v1 to v2 diff --git a/docs/documentation/architecture-and-internals.md b/docs/documentation/architecture-and-internals.md new file mode 100644 index 0000000000..066d2d2356 --- /dev/null +++ b/docs/documentation/architecture-and-internals.md @@ -0,0 +1,18 @@ +--- +title: Architecture and Internals +description: Architecture and Internals for Developers +layout: docs +permalink: /docs/architecture-and-internals +--- + +# Architecture and Internals + +This document gives an overview of the internal structure and components of Java Operator SDK core, in order to +make it easier for developers to understand and contribute to it. + + +## The Big Picture and Core Components + + + + From dbc76d1cc8ef45a39196079fe7a9ebbab556c046 Mon Sep 17 00:00:00 2001 From: csviri Date: Thu, 16 Dec 2021 15:04:53 +0100 Subject: [PATCH 2/3] feature: architecture diagram --- docs/assets/images/architecture.svg | 4 ++ .../architecture-and-internals.md | 42 ++++++++++++++++--- 2 files changed, 40 insertions(+), 6 deletions(-) create mode 100644 docs/assets/images/architecture.svg diff --git a/docs/assets/images/architecture.svg b/docs/assets/images/architecture.svg new file mode 100644 index 0000000000..0f5a97da0d --- /dev/null +++ b/docs/assets/images/architecture.svg @@ -0,0 +1,4 @@ + + + +
Operator
Operator
Controller
Controller
EventSourceManager
EventSourceManager
EventProcessor
EventProcessor
ReconcilerDispatcher
ReconcilerDispatcher
EventSource
EventSource
Propagate Event
Propagate Event
0..*
0..*
1..*
1..*
ControllerResourceEventSource
ControllerResourceEventSource
Propagate Event
Propagate Event
Reconciler
ReconcilerViewer does not support full SVG 1.1 \ No newline at end of file diff --git a/docs/documentation/architecture-and-internals.md b/docs/documentation/architecture-and-internals.md index 066d2d2356..692ea1ba43 100644 --- a/docs/documentation/architecture-and-internals.md +++ b/docs/documentation/architecture-and-internals.md @@ -7,12 +7,42 @@ permalink: /docs/architecture-and-internals # Architecture and Internals -This document gives an overview of the internal structure and components of Java Operator SDK core, in order to -make it easier for developers to understand and contribute to it. - +This document gives an overview of the internal structure and components of Java Operator SDK core, in order to make it +easier for developers to understand and contribute to it. However, this is just an extract of the backbone of the core +module, but other parts should be fairly easy to understand. We will maintain this document on developer feedback. ## The Big Picture and Core Components - - - +![Alt text for broken image link](../assets/images/architecture.svg) + +[Operator](https://github.com/java-operator-sdk/java-operator-sdk/blob/main/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/Operator.java) +is a set of +independent [controllers](https://github.com/java-operator-sdk/java-operator-sdk/blob/main/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/Controller.java) +. Controller however, is an internal class managed by the framework itself. It encapsulates directly or indirectly all +the processing units for a single custom resource. Other components: + +- [EventSourceManager](https://github.com/java-operator-sdk/java-operator-sdk/blob/main/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/EventSourceManager.java) + aggregates all the event sources regarding a controller. Provides starts and stops the event sources. +- [ControllerResourceEventSource](https://github.com/java-operator-sdk/java-operator-sdk/blob/main/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/controller/ControllerResourceEventSource.java) + is a central event source that watches the controller related custom resource for changes, propagates events and caches the + state of the custom resources. In the background from V2 it uses Informers. +- [EventProcessor](https://github.com/java-operator-sdk/java-operator-sdk/blob/main/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/EventProcessor.java) + processes the incoming events. Implements execution serialization. Manages the executor service for execution. Also implements the post-processing + of after the reconciler was executed, like re-schedules and retries of events. +- [ReconcilerDispatcher](https://github.com/java-operator-sdk/java-operator-sdk/blob/main/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/ReconciliationDispatcher.java) + is responsible for managing logic around reconciler execution, deciding which method should be called of the reconciler, managing the result + (UpdateControl and DeleteControl), making the instructed Kubernetes API calls. +- [Reconciler](https://github.com/java-operator-sdk/java-operator-sdk/blob/main/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/Reconciler.java) + is the primary entry-point for the developers of the framework to implement the reconciliation logic. + +## Typical Workflow + +A typical workflows looks like following: +1. An EventSource produces and event, that is propagated to the event processor. +2. In the event processor the related `CustomResource` is read from the cache based on the `ResourceID` in the event. +3. If there is no other execution running for the custom resource, an execution is submitted for the executor (thread pool) . +4. Executor call EventDispatcher what decides which method to execute of the reconciler. Let's say in this case it was `reconcile(...)` +5. After reconciler execution the Dispatcher calls Kubernetes API server, since the `reconcile` method returned with `UpdateControl.updateStatus(...)` result. +6. Now the dispatcher finishes the execution and calls back `EventProcessor` to finalize the execution. +7. EventProcessor checks if there is no `reschedule` or `retry` required and if there are no subsequent events received for the custom resource +8. Neither of this happened, therefore the event execution finished. From ce42cc2da9ddf2155d00e94f4e571e38c01c6d4c Mon Sep 17 00:00:00 2001 From: csviri Date: Thu, 16 Dec 2021 15:05:34 +0100 Subject: [PATCH 3/3] fix: fomratting of md --- .../architecture-and-internals.md | 30 +++++++++++-------- 1 file changed, 18 insertions(+), 12 deletions(-) diff --git a/docs/documentation/architecture-and-internals.md b/docs/documentation/architecture-and-internals.md index 692ea1ba43..7163824a3d 100644 --- a/docs/documentation/architecture-and-internals.md +++ b/docs/documentation/architecture-and-internals.md @@ -24,25 +24,31 @@ the processing units for a single custom resource. Other components: - [EventSourceManager](https://github.com/java-operator-sdk/java-operator-sdk/blob/main/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/EventSourceManager.java) aggregates all the event sources regarding a controller. Provides starts and stops the event sources. - [ControllerResourceEventSource](https://github.com/java-operator-sdk/java-operator-sdk/blob/main/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/controller/ControllerResourceEventSource.java) - is a central event source that watches the controller related custom resource for changes, propagates events and caches the - state of the custom resources. In the background from V2 it uses Informers. + is a central event source that watches the controller related custom resource for changes, propagates events and + caches the state of the custom resources. In the background from V2 it uses Informers. - [EventProcessor](https://github.com/java-operator-sdk/java-operator-sdk/blob/main/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/EventProcessor.java) - processes the incoming events. Implements execution serialization. Manages the executor service for execution. Also implements the post-processing - of after the reconciler was executed, like re-schedules and retries of events. + processes the incoming events. Implements execution serialization. Manages the executor service for execution. Also + implements the post-processing of after the reconciler was executed, like re-schedules and retries of events. - [ReconcilerDispatcher](https://github.com/java-operator-sdk/java-operator-sdk/blob/main/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/ReconciliationDispatcher.java) - is responsible for managing logic around reconciler execution, deciding which method should be called of the reconciler, managing the result - (UpdateControl and DeleteControl), making the instructed Kubernetes API calls. + is responsible for managing logic around reconciler execution, deciding which method should be called of the + reconciler, managing the result + (UpdateControl and DeleteControl), making the instructed Kubernetes API calls. - [Reconciler](https://github.com/java-operator-sdk/java-operator-sdk/blob/main/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/Reconciler.java) is the primary entry-point for the developers of the framework to implement the reconciliation logic. ## Typical Workflow A typical workflows looks like following: + 1. An EventSource produces and event, that is propagated to the event processor. -2. In the event processor the related `CustomResource` is read from the cache based on the `ResourceID` in the event. -3. If there is no other execution running for the custom resource, an execution is submitted for the executor (thread pool) . -4. Executor call EventDispatcher what decides which method to execute of the reconciler. Let's say in this case it was `reconcile(...)` -5. After reconciler execution the Dispatcher calls Kubernetes API server, since the `reconcile` method returned with `UpdateControl.updateStatus(...)` result. -6. Now the dispatcher finishes the execution and calls back `EventProcessor` to finalize the execution. -7. EventProcessor checks if there is no `reschedule` or `retry` required and if there are no subsequent events received for the custom resource +2. In the event processor the related `CustomResource` is read from the cache based on the `ResourceID` in the event. +3. If there is no other execution running for the custom resource, an execution is submitted for the executor (thread + pool) . +4. Executor call EventDispatcher what decides which method to execute of the reconciler. Let's say in this case it + was `reconcile(...)` +5. After reconciler execution the Dispatcher calls Kubernetes API server, since the `reconcile` method returned + with `UpdateControl.updateStatus(...)` result. +6. Now the dispatcher finishes the execution and calls back `EventProcessor` to finalize the execution. +7. EventProcessor checks if there is no `reschedule` or `retry` required and if there are no subsequent events received + for the custom resource 8. Neither of this happened, therefore the event execution finished.