v0.44.0 logs not json

[haarchri]

What happened?

with v0.44.0 we can see new log messages - these log messages not logged as json

2023/11/18 14:36:17 [WARN] EKS Add-On (platform-ref-aws-cnoe-nm9pn-9l46l:aws-ebs-csi-driver) not found, removing from state
2023/11/18 14:36:17 [DEBUG] Waiting for state to become: [success]
2023/11/18 14:36:19 [WARN] EKS Add-On (platform-ref-aws-cnoe-nm9pn-9l46l:aws-ebs-csi-driver) not found, removing from state
2023/11/18 14:36:21 [WARN] EKS Add-On (platform-ref-aws-cnoe-nm9pn-9l46l:aws-ebs-csi-driver) not found, removing from state
2023/11/18 14:36:23 [WARN] EKS Add-On (platform-ref-aws-cnoe-nm9pn-9l46l:aws-ebs-csi-driver) not found, removing from state
2023/11/18 14:36:30 [WARN] EKS Add-On (platform-ref-aws-cnoe-nm9pn-9l46l:aws-ebs-csi-driver) not found, removing from state
2023/11/18 14:36:32 [WARN] EKS Add-On (platform-ref-aws-cnoe-nm9pn-9l46l:aws-ebs-csi-driver) not found, removing from state
2023/11/18 14:37:06 [WARN] EKS Add-On (platform-ref-aws-cnoe-nm9pn-9l46l:aws-ebs-csi-driver) not found, removing from state
2023/11/18 14:37:06 [DEBUG] Waiting for state to become: [success]
2023/11/18 14:37:08 [WARN] EKS Add-On (platform-ref-aws-cnoe-nm9pn-9l46l:aws-ebs-csi-driver) not found, removing from state
2023/11/18 14:37:10 [WARN] EKS Add-On (platform-ref-aws-cnoe-nm9pn-9l46l:aws-ebs-csi-driver) not found, removing from state
2023/11/18 14:38:10 [WARN] EKS Add-On (platform-ref-aws-cnoe-nm9pn-9l46l:aws-ebs-csi-driver) not found, removing from state
2023/11/18 14:38:10 [DEBUG] Waiting for state to become: [success]
2023/11/18 14:38:11 [WARN] EKS Add-On (platform-ref-aws-cnoe-nm9pn-9l46l:aws-ebs-csi-driver) not found, removing from state
2023/11/18 14:38:13 [WARN] EKS Cluster (platform-ref-aws-cnoe-nm9pn-9l46l) not found, removing from state
2023/11/18 14:38:13 [WARN] EKS Add-On (platform-ref-aws-cnoe-nm9pn-9l46l:aws-ebs-csi-driver) not found, removing from state
2023/11/18 14:38:15 [WARN] EKS Cluster (platform-ref-aws-cnoe-nm9pn-9l46l) not found, removing from state
2023/11/18 14:38:16 [WARN] EKS Cluster (platform-ref-aws-cnoe-nm9pn-9l46l) not found, removing from state
2023/11/18 14:38:18 [WARN] EKS Cluster (platform-ref-aws-cnoe-nm9pn-9l46l) not found, removing from state
2023/11/18 14:38:20 [WARN] EKS Cluster (platform-ref-aws-cnoe-nm9pn-9l46l) not found, removing from state
2023/11/18 14:38:21 [WARN] EKS Cluster (platform-ref-aws-cnoe-nm9pn-9l46l) not found, removing from state
2023/11/18 14:38:23 [WARN] EKS Cluster (platform-ref-aws-cnoe-nm9pn-9l46l) not found, removing from state
2023/11/18 14:38:25 [WARN] EKS Cluster (platform-ref-aws-cnoe-nm9pn-9l46l) not found, removing from state
2023/11/18 14:38:27 [WARN] EKS Cluster (platform-ref-aws-cnoe-nm9pn-9l46l) not found, removing from state
2023/11/18 14:38:28 [WARN] EKS Cluster (platform-ref-aws-cnoe-nm9pn-9l46l) not found, removing from state
2023/11/18 14:38:30 [WARN] EKS Cluster (platform-ref-aws-cnoe-nm9pn-9l46l) not found, removing from state
2023/11/18 14:38:32 [WARN] EKS Cluster (platform-ref-aws-cnoe-nm9pn-9l46l) not found, removing from state
2023/11/18 14:38:33 [WARN] EKS Cluster (platform-ref-aws-cnoe-nm9pn-9l46l) not found, removing from state
2023/11/18 14:38:35 [WARN] EKS Cluster (platform-ref-aws-cnoe-nm9pn-9l46l) not found, removing from state
2023/11/18 14:38:37 [WARN] EKS Cluster (platform-ref-aws-cnoe-nm9pn-9l46l) not found, removing from state
2023/11/18 14:38:39 [WARN] EKS Cluster (platform-ref-aws-cnoe-nm9pn-9l46l) not found, removing from state
2023/11/18 14:38:40 [WARN] EKS Cluster (platform-ref-aws-cnoe-nm9pn-9l46l) not found, removing from state
2023/11/18 14:38:42 [WARN] EKS Cluster (platform-ref-aws-cnoe-nm9pn-9l46l) not found, removing from state

How can we reproduce it?

What environment did it happen in?

• Crossplane Version:
• Provider Version:
• Kubernetes Version:
• Kubernetes Distribution:

[sidpalas]

I'm still seeing this in v0.47.1.

Most of these non JSON logs im seeing look like they may be coming from the terraform-plugin-sdk or terraform-provider-aws

The most prevalent in my setup is [DEBUG] Waiting for state to become: [success]

https://github.com/hashicorp/terraform-plugin-sdk/blame/f2441c8e36b5d3ff3fbf3f7d4df15a8e69e9326b/helper/retry/state.go#L58

Or for the one you showed above: [WARN] EKS Cluster ...

https://github.com/hashicorp/terraform-provider-aws/blame/a8fa674da96e16e11fd3a0591bcc3d0241ff75c2/internal/service/eks/addon.go#L213

I am not sure the full machinery of how upjet uses the terrafrorm-plugin-sdk and terraform-provider-aws but presumably something changed about how the log level gets configured during the code gen process.