Skip to content

Computeenvironment and jobqueue - #1542

Merged
jeanduplessis merged 5 commits into
crossplane-contrib:mainfrom
sachinmalanki:computeenv-jobqueue
Oct 29, 2024
Merged

Computeenvironment and jobqueue#1542
jeanduplessis merged 5 commits into
crossplane-contrib:mainfrom
sachinmalanki:computeenv-jobqueue

Conversation

@sachinmalanki

@sachinmalanki sachinmalanki commented Oct 25, 2024

Copy link
Copy Markdown
Contributor

Description of your changes

Creating new AWS Batch Resources - ComputeEnvironment and JobQueue
Continuing from - #1489

I have:

  • Read and followed Crossplane's contribution process.
  • Run make reviewable to ensure this PR is ready for review.
  • Added backport release-x.y labels to auto-backport this PR if necessary.

How has this code been tested

This code has been tested locally and with uptest in the below comments.

@sachinmalanki

Copy link
Copy Markdown
Contributor Author

/test-examples="examples/batch/v1beta1/computeenvironment.yaml"

3 similar comments
@turkenf

turkenf commented Oct 25, 2024

Copy link
Copy Markdown
Collaborator

/test-examples="examples/batch/v1beta1/computeenvironment.yaml"

@sachinmalanki

Copy link
Copy Markdown
Contributor Author

/test-examples="examples/batch/v1beta1/computeenvironment.yaml"

@turkenf

turkenf commented Oct 25, 2024

Copy link
Copy Markdown
Collaborator

/test-examples="examples/batch/v1beta1/computeenvironment.yaml"

@sachinmalanki

Copy link
Copy Markdown
Contributor Author

The Uptest fails because of this error.
logger.go:42: 13:57:17 | case/0-apply | - lastTransitionTime: "2024-10-25T13:56:19Z" logger.go:42: 13:57:17 | case/0-apply | message: 'update failed: async update failed: failed to set observation: v1beta1.ComputeEnvironmentObservation.ComputeResources: logger.go:42: 13:57:17 | case/0-apply | readObjectStart: expect { or n, but found [, error found in #10 byte of ...|sources":[{"allocati|..., logger.go:42: 13:57:17 | case/0-apply | bigger context ...|,"id":"sample","region":null,"compute_resources":[{"allocation_strategy":"","desired_vcpus":0,"max_v|...' logger.go:42: 13:57:17 | case/0-apply | reason: ReconcileError logger.go:42: 13:57:17 | case/0-apply | status: "False" logger.go:42: 13:57:17 | case/0-apply | type: Synced logger.go:42: 13:57:17 | case/0-apply | - lastTransitionTime: "2024-10-25T13:42:44Z" logger.go:42: 13:57:17 | case/0-apply | reason: Available logger.go:42: 13:57:17 | case/0-apply | status: "True" logger.go:42: 13:57:17 | case/0-apply | type: Ready logger.go:42: 13:57:17 | case/0-apply | - lastTransitionTime: "2024-10-25T13:56:19Z" logger.go:42: 13:57:17 | case/0-apply | message: 'async update failed: failed to set observation: v1beta1.ComputeEnvironmentObservation.ComputeResources: logger.go:42: 13:57:17 | case/0-apply | readObjectStart: expect { or n, but found [, error found in #10 byte of ...|sources":[{"allocati|..., logger.go:42: 13:57:17 | case/0-apply | bigger context ...|,"id":"sample","region":null,"compute_resources":[{"allocation_strategy":"","desired_vcpus":0,"max_v|...' logger.go:42: 13:57:17 | case/0-apply | reason: AsyncUpdateFailure logger.go:42: 13:57:17 | case/0-apply | status: "False" logger.go:42: 13:57:17 | case/0-apply | type: LastAsyncOperation

Any idea on how to fix this? I can see that compute resource expects object but is provided an array which is likely the reason for this error.

Comment thread config/batch/config.go
Comment on lines +9 to +21
func Configure(p *config.Provider) {
p.AddResourceConfigurator("aws_batch_compute_environment", func(r *config.Resource) {
r.References = config.References{
"compute_resources.instance_role": config.Reference{
TerraformName: "aws_iam_instance_profile",
},
"compute_resources.placement_group": config.Reference{
TerraformName: "aws_placement_group",
},
"service_role": config.Reference{
TerraformName: "aws_iam_role",
},
"compute_resources.subnets": config.Reference{
TerraformName: "aws_subnet",
},
"compute_resources.security_group_ids": config.Reference{
TerraformName: "aws_security_group",
},
}
r.UseAsync = true
})

p.AddResourceConfigurator("aws_batch_job_queue", func(r *config.Resource) {
r.References = config.References{
"compute_environment_order.compute_environment": config.Reference{
TerraformName: "aws_batch_compute_environment",
Extractor: common.PathARNExtractor,
},
}
})
}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested these two resources in my local, and except for the automatically defined references, defining the above seems sufficient. Could you please remove the above and add only the following configurations?

func Configure(p *config.Provider) {
	p.AddResourceConfigurator("aws_batch_compute_environment", func(r *config.Resource) {
		r.References["compute_resources.subnets"] = config.Reference{
			TerraformName: "aws_subnet",
		}
		r.References["compute_resources.security_group_ids"] = config.Reference{
			TerraformName: "aws_security_group",
		}
	})
}

Comment thread examples/batch/v1beta1/jobqueue.yaml Outdated
Comment on lines +19 to +20
providerConfigRef:
name: aws-provider-config-test

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We use the default provider config in uptest; could you please remove the above?

Comment thread examples/batch/v1beta1/jobqueue.yaml Outdated
Comment on lines +54 to +55
providerConfigRef:
name: aws-provider-config-test

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We use the default provider config in uptest; could you please remove the above?

@turkenf

turkenf commented Oct 28, 2024

Copy link
Copy Markdown
Collaborator

Could you please also remove the batch group from externalnamenottested.go

Thank you 🙏

Signed-off-by: sachinmalanki <sachin.chini@gmail.com>
@turkenf

turkenf commented Oct 28, 2024

Copy link
Copy Markdown
Collaborator

/test-examples="examples/batch/v1beta1/computeenvironment.yaml"

@turkenf

turkenf commented Oct 28, 2024

Copy link
Copy Markdown
Collaborator

/test-examples="examples/iot/v1beta1/topicruledestination.yaml"

Signed-off-by: sachinmalanki <sachin.chini@gmail.com>
@turkenf
turkenf force-pushed the computeenv-jobqueue branch from ce668b3 to 33d589a Compare October 28, 2024 17:18
@turkenf

turkenf commented Oct 28, 2024

Copy link
Copy Markdown
Collaborator

/test-examples="examples/batch/v1beta1/computeenvironment.yaml"

@turkenf
turkenf force-pushed the computeenv-jobqueue branch from 33d589a to e5a4733 Compare October 28, 2024 17:34
@turkenf

turkenf commented Oct 28, 2024

Copy link
Copy Markdown
Collaborator

/test-examples="examples/batch/v1beta1/computeenvironment.yaml"

Signed-off-by: sachinmalanki <sachin.chini@gmail.com>
@turkenf

turkenf commented Oct 28, 2024

Copy link
Copy Markdown
Collaborator

/test-examples="examples/batch/v1beta1/computeenvironment.yaml"

@turkenf

turkenf commented Oct 28, 2024

Copy link
Copy Markdown
Collaborator

/test-examples="examples/batch/v1beta1/jobqueue.yaml"

Signed-off-by: sachinmalanki <sachin.chini@gmail.com>
@turkenf

turkenf commented Oct 28, 2024

Copy link
Copy Markdown
Collaborator

/test-examples="examples/batch/v1beta1/computeenvironment.yaml"

Uptest run: https://github.com/crossplane-contrib/provider-upjet-aws/actions/runs/11562496508

@turkenf

turkenf commented Oct 28, 2024

Copy link
Copy Markdown
Collaborator

/test-examples="examples/batch/v1beta1/jobqueue.yaml"

Uptest run: https://github.com/crossplane-contrib/provider-upjet-aws/actions/runs/11562654341

@turkenf turkenf left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Many thanks for your effort and quick actions @sachinmalanki. The tests were finally passed successfully 🎉

Finally, a few cosmetic corrections would be nice before moving forward. I left a few comments for your consideration.

Comment thread examples/batch/v1beta1/jobqueue.yaml Outdated
Comment thread examples/batch/v1beta1/testhooks/delete-computeenvironment.sh Outdated
Comment thread examples/batch/v1beta1/computeenvironment.yaml Outdated
Signed-off-by: sachinmalanki <sachin.chini@gmail.com>
@sachinmalanki

Copy link
Copy Markdown
Contributor Author

Thanks @turkenf for your help. I have made the small changes suggested

@jeanduplessis
jeanduplessis merged commit 0016fd6 into crossplane-contrib:main Oct 29, 2024
ido-re pushed a commit to ido-re/provider-upjet-aws that referenced this pull request Mar 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants