test(plugin-iceberg): Test for alter column types for Iceberg#27744
Closed
hantangwangd wants to merge 4 commits into
Closed
test(plugin-iceberg): Test for alter column types for Iceberg#27744hantangwangd wants to merge 4 commits into
hantangwangd wants to merge 4 commits into
Conversation
Contributor
Reviewer's GuideImplements ALTER TABLE ... ALTER COLUMN ... SET DATA TYPE support end-to-end (parser, analyzer, execution, metadata, access control, connector plumbing) and adds Iceberg integration and tests, including type-coercion handling for REAL→DOUBLE and DECIMAL widening across ORC/Parquet, plus new stats wiring and utility fixes. Sequence diagram for ALTER TABLE ALTER COLUMN SET DATA TYPE execution pathsequenceDiagram
actor User
participant Coordinator
participant SqlParser
participant AstBuilder
participant StatementAnalyzer
participant ExecutionPlanner
participant SetColumnTypeTask
participant AccessControlManager
participant MetadataManager
participant ConnectorMetadata
participant IcebergMetadata
User->>Coordinator: submit ALTER TABLE t ALTER COLUMN c SET DATA TYPE new_type
Coordinator->>SqlParser: parse(sql)
SqlParser->>AstBuilder: build AST
AstBuilder-->>Coordinator: SetColumnType node
Coordinator->>StatementAnalyzer: analyze(SetColumnType)
StatementAnalyzer-->>Coordinator: analysis complete
Coordinator->>ExecutionPlanner: plan DDL(SetColumnType)
ExecutionPlanner->>SetColumnTypeTask: execute(statement, session)
SetColumnTypeTask->>MetadataManager: getMetadataResolver(session)
MetadataManager-->>SetColumnTypeTask: resolver
SetColumnTypeTask->>MetadataManager: getTableHandle(tableName)
MetadataManager-->>SetColumnTypeTask: Optional TableHandle
SetColumnTypeTask->>MetadataManager: getMaterializedView(tableName)
MetadataManager-->>SetColumnTypeTask: Optional MaterializedViewDefinition
SetColumnTypeTask->>AccessControlManager: checkCanAlterColumn(transactionId, identity, context, tableName)
AccessControlManager->>SystemAccessControl: checkCanAlterColumn(identity, context, catalogSchemaTableName)
SystemAccessControl-->>AccessControlManager: allowed or throws
AccessControlManager->>ConnectorAccessControl: checkCanAlterColumn(transactionHandle, connectorIdentity, context, schemaTableName)
ConnectorAccessControl-->>AccessControlManager: allowed or throws
AccessControlManager-->>SetColumnTypeTask: authorization result
SetColumnTypeTask->>MetadataManager: getColumnHandles(session, tableHandle)
MetadataManager-->>SetColumnTypeTask: Map columnName to ColumnHandle
SetColumnTypeTask->>MetadataManager: setColumnType(session, tableHandle, columnHandle, Type)
MetadataManager->>ConnectorMetadata: setColumnType(connectorSession, connectorTableHandle, columnHandle, Type)
ConnectorMetadata->>IcebergMetadata: setColumnType(session, tableHandle, columnHandle, Type)
IcebergMetadata->>IcebergMetadata: getIcebergTable(schemaTableName)
IcebergMetadata->>IcebergMetadata: updateSchema().updateColumn(name, icebergType).commit()
IcebergMetadata-->>ConnectorMetadata: success
ConnectorMetadata-->>MetadataManager: success
MetadataManager-->>SetColumnTypeTask: success
SetColumnTypeTask-->>ExecutionPlanner: immediateFuture(null)
ExecutionPlanner-->>Coordinator: task complete
Coordinator-->>User: DDL succeeded
Class diagram for SetColumnType statement and taskclassDiagram
class Statement {
<<abstract>>
}
class SetColumnType {
+QualifiedName tableName
+Identifier columnName
+String type
+boolean tableExists
+SetColumnType(NodeLocation location, QualifiedName tableName, Identifier columnName, String type, boolean tableExists)
+QualifiedName getTableName()
+Identifier getColumnName()
+String getType()
+boolean isTableExists()
+<R, C> R accept(AstVisitor visitor, C context)
}
class AstVisitor {
+R visitSetColumnType(SetColumnType node, C context)
}
class AstBuilder {
+Node visitSetColumnType(SqlBaseParser_SetColumnTypeContext context)
}
class SqlFormatter_Visitor {
+Void visitSetColumnType(SetColumnType node, Integer context)
}
class StatementAnalyzer_Visitor {
+Scope visitSetColumnType(SetColumnType node, Optional scope)
}
class SqlBase_grammar {
+rule setColumnType
}
class StatementUtils {
+Map statementQueryTypes
}
class PrestoDataDefBindingHelper {
+Map dataDefBuilder
}
class DDLDefinitionTask~T~ {
<<interface>>
+String getName()
}
class SetColumnTypeTask {
-Metadata metadata
+SetColumnTypeTask(Metadata metadata)
+String getName()
+ListenableFuture execute(SetColumnType statement, TransactionManager transactionManager, Metadata metadata, AccessControl accessControl, Session session, List parameters, WarningCollector warningCollector, String query)
-Type getColumnType(SetColumnType statement)
}
class Metadata {
<<interface>>
+Optional getMetadataResolver(Session session)
+Map getColumnHandles(Session session, TableHandle tableHandle)
+void setColumnType(Session session, TableHandle tableHandle, ColumnHandle column, Type type)
}
class MetadataManager {
+void setColumnType(Session session, TableHandle tableHandle, ColumnHandle column, Type type)
}
class StatsRecordingMetadataManager {
+void setColumnType(Session session, TableHandle tableHandle, ColumnHandle column, Type type)
+MetadataManagerStats stats
}
class MetadataManagerStats {
-AtomicLong setColumnTypeCalls
-TimeStat setColumnTypeTime
+void recordSetColumnTypeCall(long duration)
}
class RemoteMetadataManager {
+void setColumnType(Session session, TableHandle tableHandle, ColumnHandle column, Type type)
}
class TestingMetadata {
+void setColumnType(ConnectorSession session, ConnectorTableHandle tableHandle, ColumnHandle column, Type type)
}
Statement <|-- SetColumnType
AstVisitor <|-- AstBuilder
AstVisitor <|-- SqlFormatter_Visitor
AstVisitor <|-- StatementAnalyzer_Visitor
SqlBase_grammar --> SetColumnType : produces
AstBuilder --> SetColumnType : constructs
SqlFormatter_Visitor --> SetColumnType : formats
StatementAnalyzer_Visitor --> SetColumnType : analyzes
DDLDefinitionTask <|.. SetColumnTypeTask
SetColumnTypeTask --> Metadata : uses
SetColumnTypeTask --> AccessControl : uses
Metadata <|.. MetadataManager
MetadataManager <|.. StatsRecordingMetadataManager
Metadata <|.. RemoteMetadataManager
Metadata <|.. TestingMetadata
StatsRecordingMetadataManager --> MetadataManagerStats
Class diagram for access control and connector metadata changes for ALTER COLUMNclassDiagram
class AccessControl {
<<interface>>
+void checkCanDropColumn(TransactionId transactionId, Identity identity, AccessControlContext context, QualifiedObjectName tableName)
+void checkCanAlterColumn(TransactionId transactionId, Identity identity, AccessControlContext context, QualifiedObjectName tableName)
}
class AccessControlManager {
+void checkCanAlterColumn(TransactionId transactionId, Identity identity, AccessControlContext context, QualifiedObjectName tableName)
}
class SystemAccessControl {
<<interface>>
+void checkCanDropSchema(Identity identity, AccessControlContext context, CatalogSchemaName schema)
+void checkCanAlterColumn(Identity identity, AccessControlContext context, CatalogSchemaTableName table)
}
class AllowAllSystemAccessControl {
+void checkCanAlterColumn(Identity identity, AccessControlContext context, CatalogSchemaTableName table)
}
class FileBasedSystemAccessControl {
+void checkCanAlterColumn(Identity identity, AccessControlContext context, CatalogSchemaTableName table)
}
class StatsRecordingSystemAccessControl {
+void checkCanAlterColumn(Identity identity, AccessControlContext context, CatalogSchemaTableName table)
+Stats stats
}
class StatsRecordingSystemAccessControl_Stats {
+SystemAccessControlStats checkCanAlterColumn
}
class ForwardingSystemAccessControl {
+void checkCanAlterColumn(Identity identity, AccessControlContext context, CatalogSchemaTableName table)
}
class ConnectorAccessControl {
<<interface>>
+void checkCanAlterColumn(ConnectorTransactionHandle transaction, ConnectorIdentity identity, AccessControlContext context, SchemaTableName tableName)
}
class FileBasedAccessControl {
+void checkCanAlterColumn(ConnectorTransactionHandle transaction, ConnectorIdentity identity, AccessControlContext context, SchemaTableName tableName)
}
class AllowAllAccessControl_connector {
+void checkCanAlterColumn(ConnectorTransactionHandle transaction, ConnectorIdentity identity, AccessControlContext context, SchemaTableName tableName)
}
class ReadOnlyAccessControl {
+void checkCanAlterColumn(ConnectorTransactionHandle transaction, ConnectorIdentity identity, AccessControlContext context, SchemaTableName tableName)
}
class LegacyAccessControl {
+void checkCanAlterColumn(ConnectorTransactionHandle transaction, ConnectorIdentity identity, AccessControlContext context, SchemaTableName tableName)
}
class ForwardingConnectorAccessControl {
+void checkCanAlterColumn(ConnectorTransactionHandle transactionHandle, ConnectorIdentity identity, AccessControlContext context, SchemaTableName tableName)
}
class DenyAllAccessControl {
+void checkCanAlterColumn(TransactionId transactionId, Identity identity, AccessControlContext context, QualifiedObjectName tableName)
}
class AllowAllAccessControl_spi {
+void checkCanAlterColumn(TransactionId transactionId, Identity identity, AccessControlContext context, QualifiedObjectName tableName)
}
class TestingAccessControlManager {
+void checkCanAlterColumn(TransactionId transactionId, Identity identity, AccessControlContext context, QualifiedObjectName tableName)
enum TestingPrivilegeType
}
class TestingPrivilegeType {
+ADD_COLUMN
+DROP_COLUMN
+RENAME_COLUMN
+SELECT_COLUMN
+ALTER_COLUMN
}
class AccessDeniedException {
+static void denyAlterColumn(String tableName)
+static void denyAlterColumn(String tableName, String extraInfo)
}
class ConnectorMetadata {
<<interface>>
+void setColumnType(ConnectorSession session, ConnectorTableHandle tableHandle, ColumnHandle column, Type type)
}
class ClassLoaderSafeConnectorMetadata {
+void setColumnType(ConnectorSession session, ConnectorTableHandle tableHandle, ColumnHandle columnHandle, Type type)
}
class BlackHoleMetadata {
+void setColumnType(ConnectorSession session, ConnectorTableHandle tableHandle, ColumnHandle columnHandle, Type type)
}
class IcebergAbstractMetadata {
+void setColumnType(ConnectorSession session, ConnectorTableHandle tableHandle, ColumnHandle columnHandle, Type type)
}
class TestingMetadata_connector {
+void setColumnType(ConnectorSession session, ConnectorTableHandle tableHandle, ColumnHandle column, Type type)
}
AccessControlManager ..|> AccessControl
AllowAllAccessControl_spi ..|> AccessControl
DenyAllAccessControl ..|> AccessControl
TestingAccessControlManager ..|> AccessControl
AllowAllSystemAccessControl ..|> SystemAccessControl
FileBasedSystemAccessControl ..|> SystemAccessControl
StatsRecordingSystemAccessControl ..|> SystemAccessControl
ForwardingSystemAccessControl ..|> SystemAccessControl
StatsRecordingSystemAccessControl --> StatsRecordingSystemAccessControl_Stats
FileBasedAccessControl ..|> ConnectorAccessControl
AllowAllAccessControl_connector ..|> ConnectorAccessControl
ReadOnlyAccessControl ..|> ConnectorAccessControl
LegacyAccessControl ..|> ConnectorAccessControl
ForwardingConnectorAccessControl ..|> ConnectorAccessControl
TestingAccessControlManager --> TestingPrivilegeType
ConnectorMetadata <|.. ClassLoaderSafeConnectorMetadata
ConnectorMetadata <|.. BlackHoleMetadata
ConnectorMetadata <|.. IcebergAbstractMetadata
ConnectorMetadata <|.. TestingMetadata_connector
AccessControlManager --> SystemAccessControl
AccessControlManager --> ConnectorAccessControl
SystemAccessControl --> AccessDeniedException
ConnectorAccessControl --> AccessDeniedException
ReadOnlyAccessControl --> AccessDeniedException
FileBasedAccessControl --> AccessDeniedException
ClassLoaderSafeConnectorMetadata --> ConnectorMetadata
BlackHoleMetadata --> ConnectorMetadata
IcebergAbstractMetadata --> ConnectorMetadata
TestingMetadata_connector --> ConnectorMetadata
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
Open
6 tasks
9e98473 to
f7878bc
Compare
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
[DNM] Just test for alter column types for Iceberg
Motivation and Context
Impact
Test Plan
Contributor checklist
Release Notes
Summary by Sourcery
Add support for ALTER TABLE ... ALTER COLUMN SET DATA TYPE, including metadata, access control, execution logic, and Iceberg implementation.
New Features:
Enhancements:
Tests: