feat: Hive listener integration #605

Maleware · 2025-05-27T12:57:07Z

Description

Adds listener Support

Definition of Done Checklist

Not all of these items are applicable to all PRs, the author should update this template to only leave the boxes in that are relevant
Please make sure all these things are done and tick the boxes

Author

Changes are OpenShift compatible
CRD changes approved
CRD documentation for all fields, following the style guide.
Helm chart can be installed and deployed operator works
Integration tests passed (for non trivial changes)
Changes need to be "offline" compatible

Reviewer

Code contains useful comments
Code contains useful logging statements
(Integration-)Test cases added
Documentation added or updated. Follows the style guide.
Changelog updated
Cargo.toml only contains references to git tags (not specific commits or branches)

Acceptance

Feature Tracker has been updated
Proper release label has been added
Roadmap has been updated

Maleware · 2025-05-28T07:51:52Z

=== NAME  kuttl
    harness.go:403: run tests finished
    harness.go:510: cleaning up
    harness.go:567: removing temp folder: ""
--- PASS: kuttl (2381.48s)
    --- PASS: kuttl/harness (0.00s)
        --- PASS: kuttl/harness/smoke_postgres-12.5.6_hive-3.1.3_openshift-false_s3-use-tls-false (109.29s)
        --- PASS: kuttl/harness/resources_hive-4.0.1_openshift-false (25.30s)
        --- PASS: kuttl/harness/kerberos-hdfs_postgres-12.5.6_hive-4.0.1_hdfs-latest-3.4.1_zookeeper-latest-3.9.3_krb5-1.21.1_openshift-false_kerberos-realm-PROD.MYCORP_kerberos-backend-mit (174.61s)
        --- PASS: kuttl/harness/kerberos-hdfs_postgres-12.5.6_hive-4.0.0_hdfs-latest-3.4.1_zookeeper-latest-3.9.3_krb5-1.21.1_openshift-false_kerberos-realm-PROD.MYCORP_kerberos-backend-mit (173.43s)
        --- PASS: kuttl/harness/kerberos-hdfs_postgres-12.5.6_hive-3.1.3_hdfs-latest-3.4.1_zookeeper-latest-3.9.3_krb5-1.21.1_openshift-false_kerberos-realm-PROD.MYCORP_kerberos-backend-mit (191.31s)
        --- PASS: kuttl/harness/kerberos-s3_postgres-12.5.6_hive-4.0.1_krb5-1.21.1_openshift-false_s3-use-tls-true_kerberos-realm-PROD.MYCORP_kerberos-backend-mit (109.84s)
        --- PASS: kuttl/harness/kerberos-s3_postgres-12.5.6_hive-4.0.1_krb5-1.21.1_openshift-false_s3-use-tls-false_kerberos-realm-PROD.MYCORP_kerberos-backend-mit (107.09s)
        --- PASS: kuttl/harness/kerberos-s3_postgres-12.5.6_hive-4.0.0_krb5-1.21.1_openshift-false_s3-use-tls-true_kerberos-realm-PROD.MYCORP_kerberos-backend-mit (132.82s)
        --- PASS: kuttl/harness/kerberos-s3_postgres-12.5.6_hive-4.0.0_krb5-1.21.1_openshift-false_s3-use-tls-false_kerberos-realm-PROD.MYCORP_kerberos-backend-mit (110.03s)
        --- PASS: kuttl/harness/kerberos-s3_postgres-12.5.6_hive-3.1.3_krb5-1.21.1_openshift-false_s3-use-tls-true_kerberos-realm-PROD.MYCORP_kerberos-backend-mit (119.68s)
        --- PASS: kuttl/harness/kerberos-s3_postgres-12.5.6_hive-3.1.3_krb5-1.21.1_openshift-false_s3-use-tls-false_kerberos-realm-PROD.MYCORP_kerberos-backend-mit (118.54s)
        --- PASS: kuttl/harness/upgrade_postgres-12.5.6_hive-old-3.1.3_hive-new-4.0.1_openshift-false (70.49s)
        --- PASS: kuttl/harness/cluster-operation_hive-latest-4.0.1_openshift-false (61.16s)
        --- PASS: kuttl/harness/logging_postgres-12.5.6_hive-4.0.0_openshift-false (79.92s)
        --- PASS: kuttl/harness/resources_hive-4.0.0_openshift-false (26.53s)
        --- PASS: kuttl/harness/resources_hive-3.1.3_openshift-false (26.19s)
        --- PASS: kuttl/harness/external-access_hive-latest-4.0.1_openshift-false (36.33s)
        --- PASS: kuttl/harness/orphaned-resources_hive-latest-4.0.1_openshift-false (37.07s)
        --- PASS: kuttl/harness/logging_postgres-12.5.6_hive-4.0.1_openshift-false (78.39s)
        --- PASS: kuttl/harness/smoke_postgres-12.5.6_hive-4.0.1_openshift-false_s3-use-tls-false (102.36s)
        --- PASS: kuttl/harness/logging_postgres-12.5.6_hive-3.1.3_openshift-false (87.95s)
        --- PASS: kuttl/harness/smoke_postgres-12.5.6_hive-4.0.1_openshift-false_s3-use-tls-true (96.98s)
        --- PASS: kuttl/harness/smoke_postgres-12.5.6_hive-4.0.0_openshift-false_s3-use-tls-false (102.11s)
        --- PASS: kuttl/harness/smoke_postgres-12.5.6_hive-4.0.0_openshift-false_s3-use-tls-true (100.83s)
        --- PASS: kuttl/harness/smoke_postgres-12.5.6_hive-3.1.3_openshift-false_s3-use-tls-true (103.19s)
PASS

Maleware · 2025-06-04T13:50:26Z

I might have run into stackabletech/hdfs-operator#686 during development.

I recognized that I can have an empty string in my discovery configMap from time to time. Behaviour appears to be flaky, but yet more often then not the emtpy string appears:

Expected

apiVersion: v1
data:
  HIVE: thrift://hive-postgres-s3-metastore-default.default.svc.cluster.local:9083
kind: ConfigMap
metadata:

Flaky faulty one

apiVersion: v1
data:
  HIVE: ""
kind: ConfigMap
metadata:

…tackabletech/hive-operator into feat/hive-listener-integration

Maleware · 2025-06-05T13:47:19Z

🟢

=== NAME  kuttl
    harness.go:403: run tests finished
    harness.go:510: cleaning up
    harness.go:567: removing temp folder: ""
--- PASS: kuttl (38.21s)
    --- PASS: kuttl/harness (0.00s)
        --- PASS: kuttl/harness/external-access_hive-latest-4.0.1_openshift-false (38.17s)
PASS

…l.j2 Co-authored-by: Malte Sander <[email protected]>

Co-authored-by: Nick <[email protected]>

NickLarsenNZ

See suggestions

rust/operator-binary/src/controller.rs

rust/operator-binary/src/crd/mod.rs

rust/operator-binary/src/discovery.rs

tests/templates/kuttl/smoke/80-assert.yaml

Co-authored-by: Nick <[email protected]>

…tackabletech/hive-operator into feat/hive-listener-integration

NickLarsenNZ

LGTM

maltesander

LGTM!

rust/operator-binary/src/controller.rs

Maleware · 2025-06-23T12:32:47Z

for listener.rs change:

=== NAME  kuttl
    harness.go:403: run tests finished
    harness.go:510: cleaning up
    harness.go:567: removing temp folder: ""
--- PASS: kuttl (36.64s)
    --- PASS: kuttl/harness (0.00s)
        --- PASS: kuttl/harness/external-access_hive-4.0.1_openshift-false (36.62s)
PASS

rust/operator-binary/src/discovery.rs

Co-authored-by: Nick <[email protected]>

rust/operator-binary/src/listener.rs

maltesander

LGTM!

Maleware added 8 commits May 27, 2025 14:55

WIP: Listener integration

d008047

Adding apiGroups for listeners

04558d9

Making clippy happy

4cf40c4

Working code

2af8e4a

Working test

42017f8

test definition

5ac30df

Cleaning up leftovers

de413d3

Fixing tests

e88fd8f

Adding Cargo.nix

ff8ebe0

Maleware marked this pull request as ready for review May 28, 2025 13:22

Cleaning up and renaming vars. First working example

8116051

lfrancke added this to Stackable Engineering Jun 4, 2025

lfrancke moved this to Development: In Progress in Stackable Engineering Jun 4, 2025

lfrancke assigned Maleware Jun 4, 2025

Maleware and others added 10 commits June 5, 2025 09:18

Making clippy happy

505471c

Merge branch 'main' into feat/hive-listener-integration

7074abf

remove unused input

0d1b223

Merge branch 'feat/hive-listener-integration' of https://github.com/s…

8781225

…tackabletech/hive-operator into feat/hive-listener-integration

Discovery configMap is filled properly

4b201a1

remove uneccessary files

19b0954

Refactor code, adding Cargo.lock and Cargo.nix

84e187e

Better comments

01a7329

Adding changelog

33e73b5

Adding docs

deb0ae7

Maleware changed the title ~~WIP: Listener integration~~ feat: Hive listener integration Jun 5, 2025

Making pre-commit happy

1e4ad32

Maleware moved this from Development: In Progress to Development: Waiting for Review in Stackable Engineering Jun 5, 2025

Maleware and others added 4 commits June 23, 2025 11:04

Update tests/templates/kuttl/kerberos-hdfs/70-install-access-hive.yam…

675cb1b

…l.j2 Co-authored-by: Malte Sander <[email protected]>

Update CHANGELOG.md

99eeb92

Co-authored-by: Nick <[email protected]>

Test for metrics port, rename metrics svc name function

35c8592

Trailing whitespace

43d5e38

NickLarsenNZ requested changes Jun 23, 2025

View reviewed changes

Maleware and others added 4 commits June 23, 2025 12:41

Update error to RoleListenerHasNoAddress

ff4e092

Co-authored-by: Nick <[email protected]>

Use named placeholders in discovery.rs

8567a52

Co-authored-by: Nick <[email protected]>

Using const for default listener class

8637ecd

Merge branch 'feat/hive-listener-integration' of https://github.com/s…

13ad404

…tackabletech/hive-operator into feat/hive-listener-integration

Maleware moved this from Development: In Progress to Development: In Review in Stackable Engineering Jun 23, 2025

using "" through all integration tests with shell interaction

247b450

NickLarsenNZ previously approved these changes Jun 23, 2025

View reviewed changes

NickLarsenNZ requested a review from maltesander June 23, 2025 11:34

maltesander previously approved these changes Jun 23, 2025

View reviewed changes

NickLarsenNZ reviewed Jun 23, 2025

View reviewed changes

rust/operator-binary/src/controller.rs Outdated Show resolved Hide resolved

Consolidate listener to module

546c733

Maleware dismissed stale reviews from maltesander and NickLarsenNZ via 546c733 June 23, 2025 12:31

update function to build_role_listener

1eddee0

NickLarsenNZ reviewed Jun 23, 2025

View reviewed changes

rust/operator-binary/src/discovery.rs Outdated Show resolved Hide resolved

Maleware and others added 2 commits June 23, 2025 14:42

more consistent wording

00375d1

fix: headless service function

65b795f

Co-authored-by: Nick <[email protected]>

maltesander reviewed Jun 24, 2025

View reviewed changes

rust/operator-binary/src/listener.rs Show resolved Hide resolved

maltesander approved these changes Jun 24, 2025

View reviewed changes

Maleware added this pull request to the merge queue Jun 24, 2025

Merged via the queue into main with commit 28b13d3 Jun 24, 2025
17 checks passed

Maleware deleted the feat/hive-listener-integration branch June 24, 2025 10:34

Maleware moved this from Development: In Review to Development: Done in Stackable Engineering Jun 24, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

feat: Hive listener integration #605

feat: Hive listener integration #605

Uh oh!

Maleware commented May 27, 2025 •

edited by maltesander

Loading

Uh oh!

Maleware commented May 28, 2025

Uh oh!

Maleware commented Jun 4, 2025

Uh oh!

Maleware commented Jun 5, 2025

Uh oh!

NickLarsenNZ left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

NickLarsenNZ left a comment

Uh oh!

maltesander left a comment

Uh oh!

Uh oh!

Maleware commented Jun 23, 2025

Uh oh!

Uh oh!

Uh oh!

maltesander left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

feat: Hive listener integration #605

feat: Hive listener integration #605

Uh oh!

Conversation

Maleware commented May 27, 2025 • edited by maltesander Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Definition of Done Checklist

Author

Reviewer

Acceptance

Uh oh!

Maleware commented May 28, 2025

Uh oh!

Maleware commented Jun 4, 2025

Expected

Flaky faulty one

Uh oh!

Maleware commented Jun 5, 2025

Uh oh!

NickLarsenNZ left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

NickLarsenNZ left a comment

Choose a reason for hiding this comment

Uh oh!

maltesander left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Maleware commented Jun 23, 2025

Uh oh!

Uh oh!

Uh oh!

maltesander left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Maleware commented May 27, 2025 •

edited by maltesander

Loading