feat: enable HCP deployment on external clusters by apedriza · Pull Request #989 · k0sproject/k0smotron

apedriza · 2025-04-07T10:05:58Z

extension of k0smtron API to support use external cluster for HCP. Only needed is a kubeconfig secret related to the cluster where pods will be placed

fix #969

github-actions · 2025-05-09T23:03:01Z

The PR is marked as stale since no activity has been recorded in 30 days

github-actions · 2025-06-09T23:03:16Z

The PR is marked as stale since no activity has been recorded in 30 days

apedriza · 2025-06-27T10:09:28Z

internal/controller/k0smotron.io/k0smotroncluster_controller.go

+		// the k0smotron.Cluster. This is because the owner references cannot be used in this case.
+		if !kmcScope.inClusterHCP {
+			var errors []error
+			for _, descendant := range kmcScope.getDescendants(ctx, kmc) {


This is what I like the least since ownerreferences cannot be used for a background deletion. Happy to hear other options 😃

Like we discussed, creating a "dummy" configmap or something acting as the "root" owner for everything might work. Requires some further testing and exploring so could be a separate PR, if that even works. 😂

apedriza · 2025-06-27T10:11:43Z

internal/controller/k0smotron.io/k0smotroncluster_statefulset.go

-		if foundStatefulSet.Status.ReadyReplicas == kmc.Spec.Replicas {
-			kmc.Status.Ready = true
-		}
+		return err
+	}
+	detectAndSetCurrentClusterVersion(foundStatefulSet, kmc)

-		return nil
+	if !isStatefulSetsEqual(&statefulSet, foundStatefulSet) {
+		return scope.client.Patch(ctx, &statefulSet, client.Apply, patchOpts...)
 	}

-	return err
+	if foundStatefulSet.Status.ReadyReplicas == 0 {
+		return fmt.Errorf("%w: no replicas ready yet for statefulset '%s' (%d/%d)", ErrNotReady, foundStatefulSet.GetName(), foundStatefulSet.Status.ReadyReplicas, kmc.Spec.Replicas)
+	}


I think we can set cluster status to ready if there is one replica already running instead of wait for all replicas to be ready. This way It decreases time for deploying worker nodes

makhov · 2025-07-02T07:37:47Z

api/k0smotron.io/v1beta1/k0smotroncluster_types.go

+	// HostingClusterKubeconfigRef is the reference to the kubeconfig of the hosting cluster.
+	// This kubeconfig will be used to deploy the k0s control plane.
+	//+kubebuilder:validation:Optional
+	HostingClusterKubeconfigRef *HostingClusterKubeconfigRef `json:"hostingClusterKubeconfigRef,omitempty"`


I'd call it just kubeconfigRef or kubeconfigSecretRef

apedriza · 2025-08-18T07:37:03Z

.golangci.yml

 linters:
  default: none 
  enable:
-    - depguard


After the recent changes in the golangci-lint configuration, some noisy errors appeared. I think it’s safe to remove the depguard linter for now. It deserves some investigation on how to configure it properly in follow-up changes.

Signed-off-by: apedriza <adripedriza@gmail.com>

apedriza · 2025-08-18T13:17:50Z

This is the contract fulfillment for k0smotron integrating ClusterProfile api https://github.com/kubernetes/enhancements/blob/master/keps/sig-multicluster/4322-cluster-inventory/README.md#secret-format. It is the responsibility of the user to locate, label and store with the proper key value for the kubeconfig secret accordingly to ClusterProfile API. That way we allow user use ClusterProfile API together with k0smotron but we are not strongly coupled to that API

jnummelin

LGTM in big picture

Added few nit-pick type things, feel free to address in a followup PR

jnummelin · 2025-08-19T09:49:45Z

internal/controller/bootstrap/worker_bootstrap_controller.go

 	Cluster     *clusterv1.Cluster
 }

+type hostingClusterScope struct {


dunno if this could've been "merged" with the existing scope object? This way we'd be able to pass a single scope object around instead of multiple. Might simplify the code a bit?

yes, I think using scope struct simplifies the change. I'll change it in this PR, thanks!

jnummelin · 2025-08-19T09:57:55Z

internal/controller/k0smotron.io/k0smotroncluster_controller.go

+		// the k0smotron.Cluster. This is because the owner references cannot be used in this case.
+		if !kmcScope.inClusterHCP {
+			var errors []error
+			for _, descendant := range kmcScope.getDescendants(ctx, kmc) {


Like we discussed, creating a "dummy" configmap or something acting as the "root" owner for everything might work. Requires some further testing and exploring so could be a separate PR, if that even works. 😂

jnummelin · 2025-08-19T10:07:34Z

internal/controller/k0smotron.io/k0smotroncluster_controller.go

+
+// getDescendants returns the list of resources that are associated with the k0smotron.Cluster
+// and that must be deleted when the k0smotron.Cluster is deleted.
+func (scope *kmcScope) getDescendants(ctx context.Context, kmc *km.Cluster) []client.Object {


This could be maybe enhanced to more generic approach, something like:

var descendantTypes = []client.Object{&v1.ConfigMapList{}, &v1.ServiceList{}} func (scope *kmcScope) getDescendants(ctx context.Context, kmc *km.Cluster) ([]client.Object, error) { descendants := make([]client.Object, 0) lOpt := client.MatchingLabels(kutil.DefaultK0smotronClusterLabels(kmc)) for _, dt := range descendantTypes { err := c.List(ctx, dt, lOpt) if err != nil { return nil, fmt.Errorf("failed to list descendants of type %s K0smotronControlPlane %s/%s: %w", reflect.TypeOf(variable).String(), kcp.Namespace, kcp.Name, err) } descendants = append(descendants, dt.Items...) } }

Could be a follow-up PR too

since this method is only used for the removal of resources belonging to the cluster and using that proposed dummy object for centralize the deletion is another different approach I will test both solutions is a separate PR

jnummelin · 2025-08-19T10:10:50Z

@apedriza any plans for docs on this? In a separate PR?

apedriza · 2025-08-19T10:49:09Z

@apedriza any plans for docs on this? In a separate PR?

@jnummelin I can add docs in this PR. Any ideas about what form it could take? A simple mention in the standalone documentation that this possibility exists?

jnummelin · 2025-08-19T11:08:16Z

A simple mention in the standalone documentation that this possibility exists?

yeah. plus some docs around the expected format of the secret for the access

Also I think we should add some notes on the RBAC expected for the "user" of the external client kubeconfig?

Signed-off-by: apedriza <adripedriza@gmail.com>

apedriza · 2025-08-20T07:34:26Z

Added docs and some refactoring for simplicity, as suggested. Other improvements will be addressed in a subsequent PR

kahirokunn · 2025-08-21T00:33:55Z

This is fantastic—exactly what I hoped for. Thank you so much!

apedriza force-pushed the poc-clusterprofile-for-hcp branch 6 times, most recently from 79f7d5d to 2098a74 Compare April 9, 2025 11:01

github-actions bot added Stale and removed Stale labels May 9, 2025

github-actions bot added the Stale label Jun 9, 2025

apedriza added keep Exempts issues and pull requests from stale workflow and removed Stale labels Jun 10, 2025

apedriza force-pushed the poc-clusterprofile-for-hcp branch 4 times, most recently from b312e39 to 652f9b4 Compare June 20, 2025 10:20

apedriza changed the title ~~Add clusterprofileref in order to target a cluster for HCP deployment~~ feat: enable HCP deployment on external clusters Jun 20, 2025

apedriza force-pushed the poc-clusterprofile-for-hcp branch 9 times, most recently from 82b025d to 7809516 Compare June 27, 2025 09:31

apedriza commented Jun 27, 2025

View reviewed changes

apedriza marked this pull request as ready for review June 27, 2025 10:14

apedriza requested a review from a team as a code owner June 27, 2025 10:14

apedriza added the enhancement New feature or request label Jun 27, 2025

makhov reviewed Jul 2, 2025

View reviewed changes

apedriza force-pushed the poc-clusterprofile-for-hcp branch 4 times, most recently from 1e2a463 to a100a29 Compare July 4, 2025 09:25

apedriza force-pushed the poc-clusterprofile-for-hcp branch 2 times, most recently from 2966e5f to a9eca86 Compare August 18, 2025 07:31

apedriza commented Aug 18, 2025

View reviewed changes

feat: enable HCP deployment on external clusters

01a2e8a

Signed-off-by: apedriza <adripedriza@gmail.com>

apedriza force-pushed the poc-clusterprofile-for-hcp branch from a9eca86 to 01a2e8a Compare August 18, 2025 07:44

jnummelin approved these changes Aug 19, 2025

View reviewed changes

apedriza added 2 commits August 19, 2025 15:44

Refactor client scope usage

7888229

Signed-off-by: apedriza <adripedriza@gmail.com>

Add docs

930e5ed

Signed-off-by: apedriza <adripedriza@gmail.com>

apedriza merged commit 353ea0f into k0sproject:main Aug 20, 2025
147 of 148 checks passed

kahirokunn mentioned this pull request Aug 28, 2025

Multi-Host Placement for K0smotronControlPlane: Highly available HCP across multiple dedicated host clusters with per-site LBs aggregated via DNS #1181

Closed

3 tasks

Conversation

apedriza commented Apr 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented May 9, 2025

Uh oh!

github-actions bot commented Jun 9, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

apedriza commented Aug 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jnummelin left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jnummelin commented Aug 19, 2025

Uh oh!

apedriza commented Aug 19, 2025

Uh oh!

jnummelin commented Aug 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

apedriza commented Aug 20, 2025

Uh oh!

Uh oh!

kahirokunn commented Aug 21, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

apedriza commented Apr 7, 2025 •

edited

Loading

apedriza commented Aug 18, 2025 •

edited

Loading

jnummelin commented Aug 19, 2025 •

edited

Loading