Skip to content

[FLINK-37913][table] Add built-in OBJECT_OF function #26704

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 23 commits into from
Jul 17, 2025

Conversation

raminqaf
Copy link
Contributor

What is the purpose of the change

This pull request implements the OBJECT_OF function as specified in FLIP-520 to enable creation of structured types in SQL and Table API. The function allows users to construct structured objects from key-value pairs without requiring UDFs, making structured type creation SQL-serializable and improving the overall usability of structured types in Flink.

Brief change log

  • Added OBJECT_OF function definition to BuiltInFunctionDefinitions
  • Implemented ObjectOfInputTypeStrategy for input validation (odd argument count, string keys, unique field names)
  • Implemented ObjectOfTypeStrategy for output type inference (creates structured types)
  • Added ObjectOfFunction runtime implementation that creates RowData from key-value pairs
  • Enhanced Expressions.java with objectOf() methods for Table API support
  • Added Python API support with object_of() function in expressions.py
  • Updated SQL functions documentation with OBJECT_OF function details

Verifying this change

This change added tests and can be verified as follows:

  • Added unit tests for ObjectOfInputTypeStrategy to validate argument count, type validation, and field name uniqueness
  • Added integration tests in StructuredFunctionsITCase to test end-to-end functionality
  • Verified SQL usage: SELECT OBJECT_OF('com.example.User', 'name', 'Bob', 'age', 42)
  • Verified Table API usage: objectOf(User.class, "name", "Bob", "age", 42)

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): no
  • The public API, i.e., is any changed class annotated with @Public(Evolving): yes (additions to Expressions.java and BuiltInFunctionDefinitions)
  • The serializers: no
  • The runtime per-record code paths (performance sensitive): no
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
  • The S3 file system connector: no

Documentation

  • Does this pull request introduce a new feature? yes
  • If yes, how is the feature documented? docs / JavaDocs (updated SQL functions documentation, comprehensive JavaDocs for all new classes and methods, Python API documentation)

@flinkbot
Copy link
Collaborator

flinkbot commented Jun 19, 2025

CI report:

Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

@gustavodemorais
Copy link
Contributor

I think there's a checkstyle violation making the CI fail. Make sure to run ./mvnw checkstyle:check locally

@raminqaf raminqaf force-pushed the FLINK-37913 branch 2 times, most recently from 8f915c5 to bd655e0 Compare June 21, 2025 14:10
@raminqaf
Copy link
Contributor Author

@twalthr There is also another case that I have prepared, but so far, it has been stashed on my branch. In the FLIP-520, you mentioned under the table API, this signature:

Expressions.objectOf(DataType, Object... kv);

In SQL, this would be equivalent to:

SELECT OBJECT_OF(OBJECT_OF(...)));

Is this a case we should also support in SQL? Does this mean that we can construct structure types from structured types?

Copy link
Contributor

@twalthr twalthr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for addressing all comments. LGTM.

@github-actions github-actions bot added community-reviewed PR has been reviewed by the community. and removed community-reviewed PR has been reviewed by the community. labels Jul 17, 2025
@twalthr twalthr merged commit 71f6717 into apache:master Jul 17, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
community-reviewed PR has been reviewed by the community.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants