Skip to content

Add aws_opensearchserverless_collection_group resource - #2160

Merged
jonasz-lasut merged 5 commits into
crossplane-contrib:mainfrom
sv-oss:feat/aoss-collection-group
Jul 31, 2026
Merged

Add aws_opensearchserverless_collection_group resource#2160
jonasz-lasut merged 5 commits into
crossplane-contrib:mainfrom
sv-oss:feat/aoss-collection-group

Conversation

@fed-sv

@fed-sv fed-sv commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Description of your changes

Adds support for the aws_opensearchserverless_collection_group resource,
which is available in the pinned Terraform provider (v6.53.0) but was not yet
registered.

Collection groups carry the capacity limits (min/max indexing and search
OCUs) and the generation setting (CLASSIC/NEXTGEN) for OpenSearch
Serverless; NextGen groups with a minimum capacity of 0 OCU are the mechanism
for OpenSearch Serverless scale-to-zero. Registering the resource also causes
the existing Collection resource to pick up generated cross-resource
references for its collectionGroupName field
(collectionGroupNameRef/collectionGroupNameSelector), since the referenced
kind now exists.

Configuration notes:

  • The resource is a Terraform Plugin Framework resource, so the external
    name config is registered in TerraformPluginFrameworkExternalNameConfigs.
  • Collection groups are imported by their AWS-assigned id, so the external
    name config mirrors the existing aws_opensearchserverless_collection
    one (IdentifierFromProvider with a stub id while the external name is
    unset).
  • No resource configurator is needed. capacity_limits is a max-1 list
    attribute in the Framework schema (not a block), so it generates as a
    list; this matches what make generate produces without overrides.

One behavior reviewers may want to weigh in on: this resource has a
Terraform resource identity (RegionalSingleParameterIdentity), and when a
create fails (e.g. a capacity-limit validation error from AWS), the stub
external name persists on the managed resource while the stored identity has
a null id. Every subsequent observe then fails with "Unexpected Identity
Change", which also blocks deletion; the MR can only be removed by dropping
its finalizer. The same pattern exists in the already-registered
aws_opensearchserverless_collection (and presumably other
IdentifierFromProvider Framework resources with identities), so this PR
follows the established convention rather than diverging, but a general fix
for the failed-create path may be worth tracking separately.

I have:

  • Read and followed Crossplane's contribution process.
  • Run make generate and committed the results (ideally in a separate commit).
  • Not made any manual changes to generated files, and verified this with make check-diff.

How has this code been tested

Beyond make generate from a clean tree (go build and
go test ./config/... pass), we built the opensearchserverless family
package from this branch and ran it on an EKS cluster (Crossplane 2.3,
namespaced .m. APIs, PodIdentity credentials) against a real AWS account:

  • Created a CollectionGroup with generation: NEXTGEN,
    standbyReplicas: ENABLED and capacity limits min 0 / max 16 OCU: the
    group became ACTIVE and the AWS-assigned id was observed as the external
    name.
  • Created a Collection with collectionGroupName pointing at the group:
    it joined the group and became ACTIVE; scale-to-zero behavior was
    confirmed end-to-end through the collection's data plane (idle to 0 OCU
    after ~10 minutes, ~10-20s wake on the next request).
  • Updated the group's capacity limits in place (min 0 → 2 → 0): the group
    updated without replacement, keeping the same id.
  • Deleted the composite: the collection deleted first and the group deleted
    cleanly afterwards.
  • AWS-side validation errors surface as expected (e.g. capacity values not
    in {0, 2, 4, 8, 16, multiples of 16} return the service's
    ValidationException on create).

Example manifests are provided for both cluster and namespaced APIs with a
NextGen scale-to-zero configuration, suitable for
/test-examples="examples/opensearchserverless/cluster/v1beta1/collectiongroup.yaml".

@erhancagirici

Copy link
Copy Markdown
Collaborator

/test-examples="examples/opensearchserverless/cluster/v1beta1/collectiongroup.yaml"

@erhancagirici

Copy link
Copy Markdown
Collaborator

@fed-sv thanks for the contribution! Could you rebase your PR to the latest main ? Looks like there are conflicts and I am unable rebase (possibly due to your fork's permissions)

One behavior reviewers may want to weigh in on: this resource has a
Terraform resource identity (RegionalSingleParameterIdentity), and when a
create fails (e.g. a capacity-limit validation error from AWS), the stub
external name persists on the managed resource while the stored identity has
a null id. Every subsequent observe then fails with "Unexpected Identity
Change", which also blocks deletion; the MR can only be removed by dropping
its finalizer.

That should be fixed in the latest main after #2151 . If interested, you can also verify that.

@fed-sv
fed-sv force-pushed the feat/aoss-collection-group branch from fe9930b to 6bfccdd Compare July 30, 2026 00:54
@fed-sv
fed-sv requested a review from jonasz-lasut as a code owner July 30, 2026 00:54
@fed-sv

fed-sv commented Jul 30, 2026

Copy link
Copy Markdown
Contributor Author

@erhancagirici rebased on latest main, thanks

@jonasz-lasut

Copy link
Copy Markdown
Collaborator

/test-examples="examples/opensearchserverless/cluster/v1beta1/collectiongroup.yaml"

@jonasz-lasut

Copy link
Copy Markdown
Collaborator

Hi @fed-sv E2E tests are failing due to resource misconfiguration, could you please update the example manifests?

Current state:

apiVersion: opensearchserverless.aws.upbound.io/v1beta1
kind: CollectionGroup
metadata:
  annotations:
    crossplane.io/external-create-failed: "2026-07-30T06:07:56Z"
    crossplane.io/external-create-pending: "2026-07-30T06:07:56Z"
    crossplane.io/external-create-succeeded: "2026-07-30T05:48:50Z"
    crossplane.io/external-name: stubcollectiongroup99
    meta.upbound.io/example-id: opensearchserverless/v1beta1/collectiongroup
    upjet.upbound.io/test: "true"
  creationTimestamp: "2026-07-30T05:48:49Z"
  finalizers:
  - finalizer.managedresource.crossplane.io
  generation: 2
  labels:
    testing.upbound.io/example-name: example
  name: example
  resourceVersion: "4220"
  uid: 6e415168-04b9-498a-a353-a3d226c86ea8
spec:
  deletionPolicy: Delete
  forProvider:
    capacityLimits:
    - maxIndexingCapacityInOcu: 10
      maxSearchCapacityInOcu: 10
      minIndexingCapacityInOcu: 0
      minSearchCapacityInOcu: 0
    description: Scale-to-zero collection group
    generation: NEXTGEN
    name: example-group
    region: us-west-1
    standbyReplicas: ENABLED
    tags:
      crossplane-kind: collectiongroup.opensearchserverless.aws.upbound.io
      crossplane-name: example
      crossplane-providerconfig: default
  initProvider: {}
  managementPolicies:
  - '*'
  providerConfigRef:
    name: default
status:
  atProvider: {}
  conditions:
  - lastTransitionTime: "2026-07-30T05:48:50Z"
    observedGeneration: 2
    reason: Creating
    status: "False"
    type: Ready
  - lastTransitionTime: "2026-07-30T05:49:22Z"
    message: |
      create failed: async create failed: resource creation call returned error diags: creating OpenSearch Serverless Collection Group: ID: "example-group"
      Cause: operation error OpenSearchServerless: CreateCollectionGroup, , ValidationException: Invalid value for maxIndexingCapacityInOCU. Allowed values are: 0, 2, 4, 8, 16, or any multiple of 16"
    observedGeneration: 2
    reason: ReconcileError
    status: "False"
    type: Synced
  - lastTransitionTime: "2026-07-30T05:48:50Z"
    message: |
      async create failed: resource creation call returned error diags: creating OpenSearch Serverless Collection Group: ID: "example-group"
      Cause: operation error OpenSearchServerless: CreateCollectionGroup, , ValidationException: Invalid value for maxIndexingCapacityInOCU. Allowed values are: 0, 2, 4, 8, 16, or any multiple of 16"
    reason: AsyncCreateFailure
    status: "False"
    type: LastAsyncOperation

@fed-sv

fed-sv commented Jul 30, 2026

Copy link
Copy Markdown
Contributor Author

@jonasz-lasut my bad, fixed

@jonasz-lasut

Copy link
Copy Markdown
Collaborator

/test-examples="examples/opensearchserverless/cluster/v1beta1/collectiongroup.yaml"

@jonasz-lasut jonasz-lasut left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the late change request but I've just went through terraform provider code and found add that there's a missing singleton conversion.

Please add the following resource configuration in config.go for both cluster and namespaced resource:

  p.AddResourceConfigurator("aws_opensearchserverless_collection_group", func(r *config.Resource) {
        r.AddSingletonListConversion("capacity_limits", "capacityLimits")
  })

capacity_limits is a framework.ResourceOptionalComputedListOfObjectsAttribute[capacityLimitsModel](ctx, 1, ...) in the Terraform fork, and since framework attribute size validators never surface as max_items in config/schema.json upjet does not pick it up on its own. We need to enforce the singleton conversion via config.

Examples would need an update as well.

Other than that LGTM

name: example
spec:
forProvider:
capacityLimits:

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please update to object

namespace: upbound-system
spec:
forProvider:
capacityLimits:

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please update to object

fed-sv added 5 commits July 31, 2026 11:41
Signed-off-by: Federico <github-fed-sv.unrigged670@passmail.com>
Signed-off-by: Federico <github-fed-sv.unrigged670@passmail.com>
Signed-off-by: Federico <github-fed-sv.unrigged670@passmail.com>
Signed-off-by: Federico <github-fed-sv.unrigged670@passmail.com>
Signed-off-by: Federico <github-fed-sv.unrigged670@passmail.com>
@fed-sv
fed-sv force-pushed the feat/aoss-collection-group branch from c9a9560 to a045e0b Compare July 31, 2026 05:35
@fed-sv

fed-sv commented Jul 31, 2026

Copy link
Copy Markdown
Contributor Author

@jonasz-lasut no worries, that's done, also rebased on latest main again

@jonasz-lasut

Copy link
Copy Markdown
Collaborator

/test-examples="examples/opensearchserverless/cluster/v1beta1/collectiongroup.yaml"

@jonasz-lasut jonasz-lasut left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for your contribution @fed-sv ! LGTM

@jonasz-lasut
jonasz-lasut merged commit 56e16cd into crossplane-contrib:main Jul 31, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants