feat: support instance applications

[BoxBoxJason]

Description of your changes

This PR adds the support of instance scoped (Oauth) Applications. This only works on self hosted instances.

This comes packaged with its full CRD api definition, controller lifecycle management logic, and associated clients methods.

There is an auto renew feature bundled so secrets can be updated on a regular basis if needed

Closes #321

I have:

• Read and followed Crossplane's contribution process.
• Followed the git conventional commit message format.
• Made sure all changes are covered by proper tests, reaching a coverage of at least 80% when applicable.
• Run make reviewable to ensure this PR is ready for review.
• Added backport release-x.y labels to auto-backport this PR if necessary.

How has this code been tested

I have:

• Successfully built and ran the provider locally against a kubernetes cluster.
• Successfully created, updated, and deleted resources of the types I changed / created.
• Ensured reconciliation loops for the changed / created resource complete without error.
  • Creation
  • Update
  • Deletion
• Deleted the resource on the app side to ensure the provider correctly handles
  unexpected drift. (should result in recreation of the resource if applicable)
• Updated the resource on the app side to ensure the provider correctly handles
  unexpected drift. (should result in an update of the resource if applicable)

[AyRickk]

I'm interested in this change

@henrysachs @janwillies @derbauer97 Is it good for you ?

[Copilot]

Pull request overview

Adds a new managed resource for GitLab instance-scoped OAuth Applications (self-hosted GitLab), including API types/CRDs, controller reconciliation logic, and supporting GitLab client helpers for both namespaced (instance.gitlab.m.crossplane.io) and cluster-scoped (instance.gitlab.crossplane.io) variants.

Changes:

• Introduces Application API types and generated scaffolding (deepcopy/managed interfaces/registration) for cluster + namespaced scopes.
• Adds controllers that reconcile Applications (observe via paginated list lookup; create returns secret via connection details; update implemented as delete-to-recreate).
• Adds instance application client helpers, tests, and an example manifest.

Reviewed changes

Copilot reviewed 10 out of 23 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
pkg/namespaced/controller/instance/setup.go	Registers the new namespaced Application controller in instance setup.
pkg/namespaced/controller/instance/applications/controller.go	Namespaced Application managed reconciler implementation (observe/create/update/delete).
pkg/namespaced/controller/instance/applications/controller_test.go	Unit tests for the namespaced Application controller behavior.
pkg/namespaced/clients/instance/application.go	Namespaced GitLab Applications client wrapper + helpers (options/observation/up-to-date checks).
pkg/namespaced/clients/instance/application_test.go	Unit tests for namespaced application client helpers.
pkg/cluster/controller/instance/zz_setup.go	Registers the new cluster-scoped Application controller in instance setup.
pkg/cluster/controller/instance/applications/zz_controller.go	Cluster-scoped (generated) Application controller implementation.
pkg/cluster/controller/instance/applications/zz_controller_test.go	Unit tests for the cluster-scoped Application controller behavior.
pkg/cluster/clients/instance/zz_application.go	Cluster-scoped (generated) GitLab Applications client wrapper + helpers.
pkg/cluster/clients/instance/zz_application_test.go	Unit tests for cluster-scoped application client helpers.
package/crds/instance.gitlab.m.crossplane.io_applications.yaml	Namespaced CRD for Application.
package/crds/instance.gitlab.crossplane.io_applications.yaml	Cluster-scoped CRD for Application.
examples/instance/application.yaml	Example manifest for creating a namespaced Application.
apis/namespaced/instance/v1alpha1/zz_generated.managedlist.go	Generated managed list support for namespaced ApplicationList.
apis/namespaced/instance/v1alpha1/zz_generated.managed.go	Generated managed interface helpers for namespaced Application.
apis/namespaced/instance/v1alpha1/zz_generated.deepcopy.go	Generated deepcopy implementations for namespaced Application types.
apis/namespaced/instance/v1alpha1/register.go	Registers namespaced Application types with the scheme.
apis/namespaced/instance/v1alpha1/application_types.go	Source API type definitions and kubebuilder markers for namespaced Application.
apis/cluster/instance/v1alpha1/zz_register.go	Registers cluster-scoped Application types with the scheme.
apis/cluster/instance/v1alpha1/zz_generated.managedlist.go	Generated managed list support for cluster-scoped ApplicationList.
apis/cluster/instance/v1alpha1/zz_generated.managed.go	Generated managed interface helpers for cluster-scoped Application.
apis/cluster/instance/v1alpha1/zz_generated.deepcopy.go	Generated deepcopy implementations for cluster-scoped Application types.
apis/cluster/instance/v1alpha1/zz_application_types.go	Generated cluster-scoped Application API types and markers.

Files not reviewed (13)

• apis/cluster/instance/v1alpha1/zz_application_types.go: Generated file
• apis/cluster/instance/v1alpha1/zz_generated.deepcopy.go: Generated file
• apis/cluster/instance/v1alpha1/zz_generated.managed.go: Generated file
• apis/cluster/instance/v1alpha1/zz_generated.managedlist.go: Generated file
• apis/cluster/instance/v1alpha1/zz_register.go: Generated file
• apis/namespaced/instance/v1alpha1/zz_generated.deepcopy.go: Generated file
• apis/namespaced/instance/v1alpha1/zz_generated.managed.go: Generated file
• apis/namespaced/instance/v1alpha1/zz_generated.managedlist.go: Generated file
• pkg/cluster/clients/instance/zz_application.go: Generated file
• pkg/cluster/clients/instance/zz_application_test.go: Generated file
• pkg/cluster/controller/instance/applications/zz_controller.go: Generated file
• pkg/cluster/controller/instance/applications/zz_controller_test.go: Generated file
• pkg/cluster/controller/instance/zz_setup.go: Generated file

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[markussiebert]

Thanks for this @BoxBoxJason — nice addition. I dug into the immutability/update story and want to align with you on the model before this lands. (I pushed a small follow-up commit to the branch first: fixes the double-wrapped delete error in Update() and the connection-secret doc comment, with regenerated zz_*/CRDs.)

The core constraint: there is no update API for instance applications

I checked both APIs and the client library:

• REST — the Applications API only exposes POST /applications (create), GET /applications (list), DELETE /applications/:id (delete) and POST /applications/:id/renew-secret. There is no PUT/PATCH. The update endpoint is only a stalled draft: MR gitlab-org/gitlab!151480 (still Draft since Apr 2024) against issue gitlab-org/gitlab#458617 (still open).
• GraphQL — no OAuth-application mutation exists at all. The only *ApplicationCreate mutation is cdApplicationCreate (continuous-deployment apps), which is unrelated.
• The WebUI can edit scopes in place (keeping the same id) because it uses the internal Admin::ApplicationsController#update Rails route, which is not exposed programmatically.

So in-place update is simply not reachable from a provider today, and a delete+recreate would rotate both the application_id and the secret, breaking every existing OAuth client. We should not do that silently.

Proposed model

1. Mark all spec fields immutable (name, redirectURI, scopes, confidential). I'd enforce it with a CEL rule (self == oldSelf) so the apiserver rejects edits directly, rather than relying only on a doc comment. When/if the PUT endpoint lands upstream, we can remove the immutability and implement real updates — relaxing a validation constraint is backward-compatible, so this is not a breaking change later.

2. Implement a full diff in Observe and report drift even though we can't fix it. The whole point of the controller/GitOps model is to reflect whether actual == desired. If someone changes scopes out-of-band in the WebUI, the resource should go Synced=False, not silently pretend it's fine.

   This needs a client-go bump: the pinned v2.34.0 Application response struct has no Scopes field, so we currently can't read actual scopes. v2.39.0 added it — see applications.go#L56-L63:

   type Application struct {
       ID            int64    `json:"id"`
       ApplicationID string   `json:"application_id"`
       ...
       Confidential  bool     `json:"confidential"`
       Scopes        []string `json:"scopes"`
   }

   The GET /applications response already returns scopes (it's in the docs response example), so bumping client-go is all that's needed to observe it. application_name, callback_url and confidential are already observable on v2.34.0.

3. Update returns a clear error instead of recreating, e.g. cannot update GitLab application in place: <field> is immutable in the GitLab API; delete and recreate to change it. Combined with (1), self-inflicted spec edits are rejected at the apiserver, and (2) surfaces any external drift.

Open question: the secret

The secret is only returned at creation. There is a rotation endpoint — POST /applications/:id/renew-secret (GitLab 16.11+) — but client-go does not wrap it (no method in v2.34.0 or v2.39.0; only Create/List/Delete). So rotation is possible at the REST level but would need a client-go addition or a raw call, and we'd have to decide how to trigger it (annotation? separate field?). I'd treat that as a follow-up rather than part of this PR.

What's your take on the immutable-now / observe-and-report / error-on-update model? And do you want secret rotation in scope here or split out?

[BoxBoxJason]

Hey there !

Thanks for this nice and complete review.

I did not think it was a problem to rotate the application completely when the spec was mismatched. However, as you mentionned, it will indeed break other applications that do not support "hot" reloads. So I agree with you, this is something to avoid.

Secret Rotation

About the SDK modification, it seems like they proceed with an ultra fast release cycle, I created the issue about one week ago and they only took 3 days to complete and release it: https://gitlab.com/gitlab-org/api/client-go/-/work_items/2268. I will implement the drift detection for the Scopes too, now that this is available.

I just created an issue to request to add the RenewSecret endpoint for Oauth Applications: https://gitlab.com/gitlab-org/api/client-go/-/work_items/2270.

I think we could wait another 3 days to see if they implement it fast this time too.

Then I'd like to go with the RenewalPeriodDays field in the spec, we already implemented that in some resources. For this resource, it would be an integer indicating when to rotate the application secret after creation / rotate. The application secret creation / rotation date will be stored in an annotation (or it could also in the secret that contains app id / app secret but that's not my preference).

What do you think about that ?

[BoxBoxJason]

Alright with this we should be ready to go, I made the MR for the gitlab client support of the renewal of application secret, it was accepted quickly, and released in v2.44.0.

This should settle it for this PR.
Please tell me what you think about this revised proposition @markussiebert

[markussiebert]

Thanks for the update, @BoxBoxJason — the immutability model and the scopes drift detection look good. I dug into the auto-renewal path and I think there's a blocking issue there before this can land.

Auto-renewal loops: the secret rotates on every reconcile once it first comes due.

The renewal schedule is stored in an annotation that's written both in Create and in Update. The problem is on the Update side: crossplane-runtime does not persist metadata (annotations/labels) changes made during Update — on that path it only writes the status subresource. So when a renewal fires:

1. Update calls RenewApplicationSecret and sets the next-renewal annotation in memory.
2. That annotation is never flushed to the API server.
3. The next Observe re-reads the old (past) annotation → still "due" → Update renews again.

The result is a rotation storm: the OAuth secret is re-rotated on every poll interval indefinitely, which would break every OAuth client and hammer the API. It doesn't show up in the unit tests because they call Update directly and assert the in-memory annotation, which always passes — the gap is purely at the reconciler-persistence level.

We reproduced this end-to-end against a live self-managed GitLab instance (provider running out-of-cluster against a throwaway control plane): with a renewal period configured and the schedule forced due, the connection secret's hash changed on every single poll cycle and the stored renewal date never advanced. 12 rotations in about a minute before we stopped it.

Suggested direction: tracking the schedule in status.atProvider instead of an annotation should do the trick, since status is persisted on every reconcile path (including Update), so the advanced date sticks. A few things that might help if you go that route — you know the code best, so take or leave them:

• It's probably easiest not to set the schedule in Create: the managed reconciler reverts status changes made during Create (it re-reads the object after persisting the external-name annotation), so initializing it lazily in Observe when it's unset tends to be cleaner. Seeding it to "now + period" rather than treating "unset" as "due now" also avoids an unnecessary rotation right after creation.
• Writing the advanced schedule in Update after a successful rotation works nicely, since that write is persisted via the status update at the end of the Update path.
• One small gotcha: if you keep the field inside atProvider, your Observe rebuilds that struct from the API response each time, so it helps to carry the previously-persisted value across the rebuild.

This is the same "secret you can't read back" situation we already solved for webhook token rotation in #375 (desired-side state tracked in status, written in Update), so it might be nice to mirror that approach here for consistency.

We re-tested the status-based approach with the same live setup and it settled to exactly one rotation when due, then held steady — so the pattern works. Happy to help if anything's unclear.

[BoxBoxJason]

Alright I just applied the changes,

Good catch @markussiebert about the renewal loop behaviour detection !

I believe this this latest commit properly fixes the problem. I added a ton of comments to make sure it isn't removed by mistake any time soon.

[markussiebert]

Re-reviewed against HEAD (7fa1ee0) — including a local build + test run of the application packages and a read of the pinned reconciler. Both of my earlier reviews are addressed. LGTM. 🎉

1. Immutability model — as discussed

• CEL self == oldSelf on name, redirectURI, scopes, confidential, so the apiserver rejects self-inflicted spec edits directly.
• Update returns errCannotUpdate instead of delete-recreate, so application_id/secret are never silently rotated. The earlier double-wrapped delete error is gone since Update no longer calls Delete.

2. Scopes drift detection — done

• client-go bumped to v2.44.0 (carries Application.Scopes), and IsApplicationUpToDate now compares name/redirectURI/confidential/scopes, so out-of-band changes surface as Synced=False.

3. Auto-renewal loop (my blocking concern) — fixed correctly

• The schedule now lives in status.atProvider.NextRenewalAt rather than an annotation. Observe preserves the prior value across the atProvider rebuild, lazily seeds it to now + period when unset (so no rotation immediately after creation), and treats nil as not-due.
• Update advances the schedule after a successful renew, and crossplane-runtime persists the status subresource on the Update path — so it settles to exactly one rotation when due and then holds steady. This matches what I reproduced end-to-end, and mirrors the webhook-token approach from #375. Nicely done.

Two non-blocking nits, take or leave:

• errCannotUpdate is also what surfaces on external scope drift, and its message ("delete and recreate") reads as user-action even when the change came from the WebUI. Purely cosmetic.
• Confidential when unset is treated as "don't care" (cp == nil → equal), so an out-of-band flip isn't flagged if the field is omitted. Reasonable default; maybe worth a one-line comment.