Skip to content

feat(opensearch): domain add more selector and references - #1337

Merged
ytsarev merged 1 commit into
crossplane-contrib:mainfrom
haarchri:feature/opensearch-domain-ref-selectors
Jun 4, 2024
Merged

feat(opensearch): domain add more selector and references#1337
ytsarev merged 1 commit into
crossplane-contrib:mainfrom
haarchri:feature/opensearch-domain-ref-selectors

Conversation

@haarchri

@haarchri haarchri commented Jun 1, 2024

Copy link
Copy Markdown
Member

Description of your changes

Fixes #

I have:

  • Read and followed Crossplane's contribution process.
  • Run make reviewable to ensure this PR is ready for review.
  • Added backport release-x.y labels to auto-backport this PR if necessary.

How has this code been tested

we want to migrate from community provider - this PR adds missing Selector & References:

https://marketplace.upbound.io/providers/crossplane-contrib/provider-aws/v0.48.1/resources/opensearchservice.aws.crossplane.io/Domain/v1alpha1#doc:spec-forProvider-encryptionAtRestOptions-kmsKeyIdRef

https://marketplace.upbound.io/providers/crossplane-contrib/provider-aws/v0.48.1/resources/opensearchservice.aws.crossplane.io/Domain/v1alpha1#doc:spec-forProvider-encryptionAtRestOptions-kmsKeyIdSelector

https://marketplace.upbound.io/providers/crossplane-contrib/provider-aws/v0.48.1/resources/opensearchservice.aws.crossplane.io/Domain/v1alpha1#doc:spec-forProvider-vpcOptions-securityGroupIdRefs

https://marketplace.upbound.io/providers/crossplane-contrib/provider-aws/v0.48.1/resources/opensearchservice.aws.crossplane.io/Domain/v1alpha1#doc:spec-forProvider-vpcOptions-securityGroupIdSelector

https://marketplace.upbound.io/providers/crossplane-contrib/provider-aws/v0.48.1/resources/opensearchservice.aws.crossplane.io/Domain/v1alpha1#doc:spec-forProvider-vpcOptions-subnetIdRefs

https://marketplace.upbound.io/providers/crossplane-contrib/provider-aws/v0.48.1/resources/opensearchservice.aws.crossplane.io/Domain/v1alpha1#doc:spec-forProvider-vpcOptions-subnetIdSelector

@ytsarev

ytsarev commented Jun 2, 2024

Copy link
Copy Markdown
Contributor

/test-examples="examples/opensearch/v1beta1/domain.yaml"

@haarchri

haarchri commented Jun 2, 2024

Copy link
Copy Markdown
Member Author

@ytsarev

 [{0 OpenSearch
    logger.go:42: 08:44:38 | case/0-apply |         Domain "example-domain-name" already exists  []}]'

ulucinar
ulucinar approved these changes Jun 4, 2024
View reviewed changes

@ulucinar ulucinar left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @haarchri, lgtm.

@ytsarev

ytsarev commented Jun 4, 2024

Copy link
Copy Markdown
Contributor

@haarchri can you please add ${Rand.RFC1123Subdomain} to the failing example as in https://github.com/crossplane/uptest?tab=readme-ov-file#injecting-dynamic-values-and-datasource so we would avoid clashes during the e2e test of the Domain?

@ytsarev

ytsarev commented Jun 4, 2024

Copy link
Copy Markdown
Contributor

/test-examples="examples/opensearch/v1beta1/domain.yaml"

@haarchri

haarchri commented Jun 4, 2024

Copy link
Copy Markdown
Member Author

2024-06-04T11:13:01.8825601Z logger.go:42: 11:13:01 | case/0-apply | - lastTransitionTime: "2024-06-04T11:08:55Z" 2024-06-04T11:13:01.8827013Z logger.go:42: 11:13:01 | case/0-apply | message: 'async create failed: failed to create the resource: [{0 creating OpenSearch 2024-06-04T11:13:01.8828741Z logger.go:42: 11:13:01 | case/0-apply | Domain: ValidationException: 1 validation error detected: Value ''example-domain-name-op-3j4sno06'' 2024-06-04T11:13:01.8830434Z logger.go:42: 11:13:01 | case/0-apply | at ''domainName'' failed to satisfy constraint: Member must have length less 2024-06-04T11:13:01.8831647Z logger.go:42: 11:13:01 | case/0-apply | than or equal to 28 []}]'

@haarchri
feat(opensearch): add ref/selector for opensearch domains
4ee826c 
Signed-off-by: Christopher Haar <christopher.haar@upbound.io>
@haarchri
haarchri force-pushed the feature/opensearch-domain-ref-selectors branch from 6d828cd to 4ee826c Compare June 4, 2024 12:53
@ytsarev

ytsarev commented Jun 4, 2024

Copy link
Copy Markdown
Contributor

/test-examples

@haarchri

haarchri commented Jun 4, 2024

Copy link
Copy Markdown
Member Author

@ytsarev can we test the following example ? examples/opensearch/v1beta1/domain.yaml

@ytsarev

ytsarev commented Jun 4, 2024

Copy link
Copy Markdown
Contributor

/test-examples="examples/opensearch/v1beta1/domain.yaml"

@haarchri

haarchri commented Jun 4, 2024

Copy link
Copy Markdown
Member Author

mhm - any ideas ?

    logger.go:42: 14:40:31 | case/0-apply |     - lastTransitionTime: "2024-06-04T14:40:11Z"
    logger.go:42: 14:40:31 | case/0-apply |       message: 'update failed: async update failed: failed to update the resource:
    logger.go:42: 14:40:31 | case/0-apply |         [{0 updating OpenSearch Domain (arn:aws:es:us-west-1:153891904029:domain/op-jz2ysr4s):
    logger.go:42: 14:40:31 | case/0-apply |         ValidationException: A change/update is in progress. Please wait for it to
    logger.go:42: 14:40:31 | case/0-apply |         complete before requesting another change.  []}]'

@haarchri

haarchri commented Jun 4, 2024

Copy link
Copy Markdown
Member Author

think its not realted to my change:

 kubectl describe domain.opensearch.aws.upbound.io/example
Name:         example
Namespace:    
Labels:       testing.upbound.io/example-name=example
Annotations:  crossplane.io/external-create-pending: 2024-06-04T15:36:03Z
              crossplane.io/external-create-succeeded: 2024-06-04T15:36:03Z
              crossplane.io/external-name: arn:aws:es:us-west-1:609897127049:domain/op-jqg7nlkn
              meta.upbound.io/example-id: opensearch/v1beta1/domain
              upjet.upbound.io/test: true
API Version:  opensearch.aws.upbound.io/v1beta1
Kind:         Domain
Metadata:
  Creation Timestamp:  2024-06-04T15:36:00Z
  Finalizers:
    finalizer.managedresource.crossplane.io
  Generation:        3
  Resource Version:  3355
  UID:               e2af1582-b526-40be-8064-0cec18096bf9
Spec:
  Deletion Policy:  Delete
  For Provider:
    Auto Tune Options:
      Desired State:        ENABLED
      Rollback On Disable:  NO_ROLLBACK
    Cluster Config:
      Instance Type:  m4.large.search
    Cognito Options:
      Identity Pool Id:  
      Role Arn:          
      User Pool Id:      
    Domain Endpoint Options:
      Tls Security Policy:  Policy-Min-TLS-1-2-2019-07
    Domain Name:            op-jqg7nlkn
    Ebs Options:
      Ebs Enabled:  true
      Volume Size:  10
      Volume Type:  gp2
    Encrypt At Rest:
      Enabled:       false
    Engine Version:  OpenSearch_1.0
    Node To Node Encryption:
      Enabled:  false
    Off Peak Window Options:
      Enabled:  true
      Off Peak Window:
        Window Start Time:
          Hours:  5
    Region:       us-west-1
    Snapshot Options:
      Automated Snapshot Start Hour:  0
    Software Update Options:
    Tags:
      Crossplane - Kind:            domain.opensearch.aws.upbound.io
      Crossplane - Name:            example
      Crossplane - Providerconfig:  default
  Init Provider:
  Management Policies:
    *
  Provider Config Ref:
    Name:  default
Status:
  At Provider:
    Arn:  arn:aws:es:us-west-1:609897127049:domain/op-jqg7nlkn
    Auto Tune Options:
      Desired State:        ENABLED
      Rollback On Disable:  NO_ROLLBACK
      Use Off Peak Window:  false
    Cluster Config:
      Cold Storage Options:
        Enabled:                      false
      Dedicated Master Count:         0
      Dedicated Master Enabled:       false
      Dedicated Master Type:          
      Instance Count:                 1
      Instance Type:                  m4.large.search
      Multi Az With Standby Enabled:  false
      Warm Count:                     0
      Warm Enabled:                   false
      Warm Type:                      
      Zone Awareness Enabled:         false
    Cognito Options:
      Enabled:           false
      Identity Pool Id:  
      Role Arn:          
      User Pool Id:      
    Dashboard Endpoint:  search-op-jqg7nlkn-phjcqlnsrmukuvxrg4yaixldn4.us-west-1.es.amazonaws.com/_dashboards
    Domain Endpoint Options:
      Custom Endpoint:                  
      Custom Endpoint Certificate Arn:  
      Custom Endpoint Enabled:          false
      Enforce Https:                    true
      Tls Security Policy:              Policy-Min-TLS-1-2-2019-07
    Domain Id:                          609897127049/op-jqg7nlkn
    Domain Name:                        op-jqg7nlkn
    Ebs Options:
      Ebs Enabled:  true
      Iops:         0
      Throughput:   0
      Volume Size:  10
      Volume Type:  gp2
    Encrypt At Rest:
      Enabled:        false
      Kms Key Id:     
    Endpoint:         search-op-jqg7nlkn-phjcqlnsrmukuvxrg4yaixldn4.us-west-1.es.amazonaws.com
    Engine Version:   OpenSearch_1.0
    Id:               arn:aws:es:us-west-1:609897127049:domain/op-jqg7nlkn
    Kibana Endpoint:  search-op-jqg7nlkn-phjcqlnsrmukuvxrg4yaixldn4.us-west-1.es.amazonaws.com/_plugin/kibana/
    Node To Node Encryption:
      Enabled:  false
    Off Peak Window Options:
      Enabled:  true
      Off Peak Window:
        Window Start Time:
          Hours:    5
          Minutes:  0
    Snapshot Options:
      Automated Snapshot Start Hour:  0
    Software Update Options:
      Auto Software Update Enabled:  false
    Tags:
      Crossplane - Kind:            domain.opensearch.aws.upbound.io
      Crossplane - Name:            example
      Crossplane - Providerconfig:  default
    Tags All:
      Crossplane - Kind:            domain.opensearch.aws.upbound.io
      Crossplane - Name:            example
      Crossplane - Providerconfig:  default
  Conditions:
    Last Transition Time:  2024-06-04T16:02:36Z
    Reason:                Available
    Status:                True
    Type:                  Ready
    Last Transition Time:  2024-06-04T16:15:47Z
    Reason:                ReconcileSuccess
    Status:                True
    Type:                  Synced
    Last Transition Time:  2024-06-04T15:47:33Z
    Reason:                Success
    Status:                True
    Type:                  LastAsyncOperation
Events:
  Type    Reason                   Age                   From                                                    Message
  ----    ------                   ----                  ----                                                    -------
  Normal  CreatedExternalResource  42m                   managed/opensearch.aws.upbound.io/v1beta1, kind=domain  Successfully requested creation of external resource
  Normal  UpdatedExternalResource  2m47s (x73 over 30m)  managed/opensearch.aws.upbound.io/v1beta1, kind=domain  Successfully requested update of external resource

2024-06-04T16:19:16Z	DEBUG	provider-aws	Diff detected	{"uid": "e2af1582-b526-40be-8064-0cec18096bf9", "name": "example", "gvk": "opensearch.aws.upbound.io/v1beta1, Kind=Domain", "instanceDiff": "*terraform.InstanceDiff{mu:sync.Mutex{state:0, sema:0x0}, Attributes:map[string]*terraform.ResourceAttrDiff{\"advanced_security_options.#\":*terraform.ResourceAttrDiff{Old:\"\", New:\"\", NewComputed:true, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}}, Destroy:false, DestroyDeposed:false, DestroyTainted:false, RawConfig:cty.NilVal, RawState:cty.NilVal, RawPlan:cty.NilVal, Meta:map[string]interface {}(nil)}"}
2024-06-04T16:19:16Z	DEBUG	provider-aws	Successfully requested update of external resource	{"controller": "managed/opensearch.aws.upbound.io/v1beta1, kind=domain", "request": {"name":"example"}, "uid": "e2af1582-b526-40be-8064-0cec18096bf9", "version": "3355", "external-name": "arn:aws:es:us-west-1:609897127049:domain/op-jqg7nlkn", "requeue-after": "2024-06-04T16:29:14Z"}
2024-06-04T16:19:16Z	DEBUG	provider-aws	Async update starting...	{"trackerUID": "e2af1582-b526-40be-8064-0cec18096bf9", "resourceName": "example", "gvk": "opensearch.aws.upbound.io/v1beta1, Kind=Domain", "tfID": "arn:aws:es:us-west-1:609897127049:domain/op-jqg7nlkn"}
2024-06-04T16:19:16Z	DEBUG	provider-aws	Updating the external resource	{"uid": "e2af1582-b526-40be-8064-0cec18096bf9", "name": "example", "gvk": "opensearch.aws.upbound.io/v1beta1, Kind=Domain"}
2024-06-04T16:19:20Z	DEBUG	provider-aws	Async update ended.	{"trackerUID": "e2af1582-b526-40be-8064-0cec18096bf9", "resourceName": "example", "gvk": "opensearch.aws.upbound.io/v1beta1, Kind=Domain", "error": null, "tfID": "arn:aws:es:us-west-1:609897127049:domain/op-jqg7nlkn"}
2024-06-04T16:19:20Z	DEBUG	provider-aws	Reconcile request has been requeued.	{"gvk": "opensearch.aws.upbound.io/v1beta1, Kind=Domain", "name": "example", "rateLimiterName": "", "when": "0s"}
2024-06-04T16:19:20Z	DEBUG	provider-aws	Reconciling	{"controller": "managed/opensearch.aws.upbound.io/v1beta1, kind=domain", "request": {"name":"example"}}
2024-06-04T16:19:20Z	DEBUG	provider-aws	Connecting to the service provider	{"uid": "e2af1582-b526-40be-8064-0cec18096bf9", "name": "example", "gvk": "opensearch.aws.upbound.io/v1beta1, Kind=Domain"}
2024-06-04T16:19:20Z	DEBUG	provider-aws	Observing the external resource	{"uid": "e2af1582-b526-40be-8064-0cec18096bf9", "name": "example", "gvk": "opensearch.aws.upbound.io/v1beta1, Kind=Domain"}
2024-06-04T16:19:21Z	DEBUG	provider-aws	Diff detected	{"uid": "e2af1582-b526-40be-8064-0cec18096bf9", "name": "example", "gvk": "opensearch.aws.upbound.io/v1beta1, Kind=Domain", "instanceDiff": "*terraform.InstanceDiff{mu:sync.Mutex{state:0, sema:0x0}, Attributes:map[string]*terraform.ResourceAttrDiff{\"advanced_security_options.#\":*terraform.ResourceAttrDiff{Old:\"\", New:\"\", NewComputed:true, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}}, Destroy:false, DestroyDeposed:false, DestroyTainted:false, RawConfig:cty.NilVal, RawState:cty.NilVal, RawPlan:cty.NilVal, Meta:map[string]interface {}(nil)}"}
2024-06-04T16:19:21Z	DEBUG	provider-aws	Successfully requested update of external resource	{"controller": "managed/opensearch.aws.upbound.io/v1beta1, kind=domain", "request": {"name":"example"}, "uid": "e2af1582-b526-40be-8064-0cec18096bf9", "version": "3355", "external-name": "arn:aws:es:us-west-1:609897127049:domain/op-jqg7nlkn", "requeue-after": "2024-06-04T16:29:45Z"}
2024-06-04T16:19:21Z	DEBUG	provider-aws	Async update starting...	{"trackerUID": "e2af1582-b526-40be-8064-0cec18096bf9", "resourceName": "example", "gvk": "opensearch.aws.upbound.io/v1beta1, Kind=Domain", "tfID": "arn:aws:es:us-west-1:609897127049:domain/op-jqg7nlkn"}
2024-06-04T16:19:21Z	DEBUG	provider-aws	Updating the external resource	{"uid": "e2af1582-b526-40be-8064-0cec18096bf9", "name": "example", "gvk": "opensearch.aws.upbound.io/v1beta1, Kind=Domain"}
2024-06-04T16:19:22Z	DEBUG	provider-aws	Calling the inner handler for Update event.	{"gvk": "opensearch.aws.upbound.io/v1beta1, Kind=Domain", "name": "example", "queueLength": 0}
2024-06-04T16:19:22Z	DEBUG	provider-aws	Reconciling	{"controller": "managed/opensearch.aws.upbound.io/v1beta1, kind=domain", "request": {"name":"example"}}
2024-06-04T16:19:22Z	DEBUG	provider-aws	Connecting to the service provider	{"uid": "e2af1582-b526-40be-8064-0cec18096bf9", "name": "example", "gvk": "opensearch.aws.upbound.io/v1beta1, Kind=Domain"}
2024-06-04T16:19:22Z	DEBUG	provider-aws	ongoing async operation	{"uid": "e2af1582-b526-40be-8064-0cec18096bf9", "name": "example", "gvk": "opensearch.aws.upbound.io/v1beta1, Kind=Domain", "opType": "update"}
2024-06-04T16:19:22Z	DEBUG	provider-aws	Cannot delete external resource	{"controller": "managed/opensearch.aws.upbound.io/v1beta1, kind=domain", "request": {"name":"example"}, "uid": "e2af1582-b526-40be-8064-0cec18096bf9", "version": "3846", "external-name": "arn:aws:es:us-west-1:609897127049:domain/op-jqg7nlkn", "deletion-timestamp": "2024-06-04 16:19:22 +0000 UTC", "error": "update operation that started at 2024-06-04 16:19:21.830019794 +0000 UTC m=+1214.870558596 is still running", "errorVerbose": "update operation that started at 2024-06-04 16:19:21.830019794 +0000 UTC m=+1214.870558596 is still running\ngithub.com/crossplane/upjet/pkg/controller.(*terraformPluginSDKAsyncExternal).Delete\n\tgithub.com/crossplane/upjet@v1.4.0/pkg/controller/external_async_tfpluginsdk.go:193\ngithub.com/crossplane/crossplane-runtime/pkg/reconciler/managed.(*Reconciler).Reconcile\n\tgithub.com/crossplane/crossplane-runtime@v1.16.0-rc.2.0.20240510094504-3f697876fa57/pkg/reconciler/managed/reconciler.go:954\ngithub.com/crossplane/crossplane-runtime/pkg/ratelimiter.(*Reconciler).Reconcile\n\tgithub.com/crossplane/crossplane-runtime@v1.16.0-rc.2.0.20240510094504-3f697876fa57/pkg/ratelimiter/reconciler.go:54\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\tsigs.k8s.io/controller-runtime@v0.17.3/pkg/internal/controller/controller.go:119\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\tsigs.k8s.io/controller-runtime@v0.17.3/pkg/internal/controller/controller.go:316\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\tsigs.k8s.io/controller-runtime@v0.17.3/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\tsigs.k8s.io/controller-runtime@v0.17.3/pkg/internal/controller/controller.go:227\nruntime.goexit\n\truntime/asm_arm64.s:1222"}

@ytsarev

ytsarev commented Jun 4, 2024

Copy link
Copy Markdown
Contributor

i trust we can merge the selectors and address the MR behaviour separately

@ytsarev
ytsarev merged commit 4b4cac4 into crossplane-contrib:main Jun 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ulucinar ulucinar ulucinar approved these changes
@sergenyalcin sergenyalcin Awaiting requested review from sergenyalcin sergenyalcin is a code owner
@turkenf turkenf Awaiting requested review from turkenf turkenf is a code owner
@mbbush mbbush Awaiting requested review from mbbush
+1 more reviewer
@ytsarev ytsarev ytsarev approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@haarchri @ytsarev @ulucinar