Icebrg alter table set datatype testcase fix#27707
Draft
NivinCS wants to merge 9 commits intoprestodb:masterfrom
Draft
Icebrg alter table set datatype testcase fix#27707NivinCS wants to merge 9 commits intoprestodb:masterfrom
NivinCS wants to merge 9 commits intoprestodb:masterfrom
Conversation
|
|
Contributor
Reviewer's GuideAdds full support for ALTER TABLE ... ALTER COLUMN ... SET DATA TYPE across Presto (parser, analyzer, metadata, security, connectors), including Iceberg and BlackHole, and introduces tests and stats wiring plus a new Iceberg error code and double/float handling fixes. Sequence diagram for ALTER TABLE ALTER COLUMN SET DATA TYPE executionsequenceDiagram
actor User
participant Client
participant Coordinator
participant Parser
participant Analyzer
participant SetColumnTypeTask
participant AccessControlManager
participant SystemAccessControl
participant ConnectorAccessControl
participant MetadataManager
participant ConnectorMetadata_Iceberg
participant IcebergTable
User->>Client: submit ALTER TABLE t ALTER COLUMN c SET DATA TYPE DOUBLE
Client->>Coordinator: send SQL
Coordinator->>Parser: parse SQL
Parser-->>Coordinator: SetColumnType AST
Coordinator->>Analyzer: analyze SetColumnType
Analyzer->>MetadataManager: resolve table handle
MetadataManager-->>Analyzer: TableHandle
Analyzer-->>Coordinator: analyzed statement
Coordinator->>SetColumnTypeTask: execute(SetColumnType, session)
SetColumnTypeTask->>MetadataManager: get table handle
MetadataManager-->>SetColumnTypeTask: Optional TableHandle
SetColumnTypeTask->>AccessControlManager: checkCanAlterColumn
AccessControlManager->>SystemAccessControl: checkCanAlterColumn
SystemAccessControl-->>AccessControlManager: ok
AccessControlManager->>ConnectorAccessControl: checkCanAlterColumn
ConnectorAccessControl-->>AccessControlManager: ok
AccessControlManager-->>SetColumnTypeTask: authorization ok
SetColumnTypeTask->>MetadataManager: getColumnHandles
MetadataManager-->>SetColumnTypeTask: column name to ColumnHandle
SetColumnTypeTask->>MetadataManager: setColumnType(Session, TableHandle, ColumnHandle, Type)
MetadataManager->>ConnectorMetadata_Iceberg: setColumnType(ConnectorSession, ConnectorTableHandle, ColumnHandle, Type)
ConnectorMetadata_Iceberg->>IcebergTable: updateSchema()
IcebergTable-->>ConnectorMetadata_Iceberg: UpdateSchema
ConnectorMetadata_Iceberg->>IcebergTable: updateColumn(name, newType)
ConnectorMetadata_Iceberg->>IcebergTable: commit()
IcebergTable-->>ConnectorMetadata_Iceberg: success
ConnectorMetadata_Iceberg-->>MetadataManager: ok
MetadataManager-->>SetColumnTypeTask: ok
SetColumnTypeTask-->>Coordinator: completed
Coordinator-->>Client: success
Client-->>User: statement succeeded
Class diagram for SetColumnType AST and execution taskclassDiagram
class Statement {
}
class SetColumnType {
+QualifiedName tableName
+Identifier columnName
+String type
+boolean tableExists
+QualifiedName getTableName()
+Identifier getColumnName()
+String getType()
+boolean isTableExists()
+<R,C> R accept(AstVisitor visitor, C context)
}
class AstVisitor {
+<R,C> R visitSetColumnType(SetColumnType node, C context)
}
class AstBuilder {
+Node visitSetColumnType(SqlBaseParser_SetColumnTypeContext context)
}
class SqlFormatter_FormatterVisitor {
+Void visitSetColumnType(SetColumnType node, Integer context)
}
class StatementAnalyzer_Visitor {
+Scope visitSetColumnType(SetColumnType node, Optional_scope scope)
}
class StatementUtils {
-Map statementQueryTypes
}
class SetColumnTypeTask {
-Metadata metadata
+String getName()
+ListenableFuture execute(SetColumnType statement, TransactionManager transactionManager, Metadata metadata, AccessControl accessControl, Session session, List parameters, WarningCollector warningCollector, String query)
-Type getColumnType(SetColumnType statement)
}
class Metadata {
+Optional getTableHandle(Session session, QualifiedObjectName tableName)
+Map getColumnHandles(Session session, TableHandle tableHandle)
+void setColumnType(Session session, TableHandle tableHandle, ColumnHandle column, Type type)
+Type getType(TypeSignature typeSignature)
}
class AccessControl {
+void checkCanAlterColumn(TransactionId transactionId, Identity identity, AccessControlContext context, QualifiedObjectName tableName)
}
class PrestoDataDefBindingHelper {
-Map dataDefBuilder
}
class SqlBaseParser_SetColumnTypeContext {
}
class Optional_scope {
}
class TransactionManager {
}
class Session {
+TransactionId getRequiredTransactionId()
+Identity getIdentity()
+AccessControlContext getAccessControlContext()
}
class WarningCollector {
}
class ListenableFuture {
}
class TypeSignature {
+static TypeSignature parseTypeSignature(String signature)
}
class Type {
}
class Identity {
}
class AccessControlContext {
}
class QualifiedObjectName {
}
class TableHandle {
}
class ColumnHandle {
}
class TransactionId {
}
class SemanticException {
}
class SemanticErrorCode {
}
Statement <|-- SetColumnType
AstVisitor <.. SetColumnType : accept
AstVisitor <|.. StatementAnalyzer_Visitor
AstVisitor <|.. SqlFormatter_FormatterVisitor
AstBuilder --> SetColumnType : creates
AstBuilder --> SqlBaseParser_SetColumnTypeContext
SqlFormatter_FormatterVisitor --> SetColumnType : formats
StatementAnalyzer_Visitor --> SetColumnType : analyzes
StatementUtils --> SetColumnType : mapsToQueryType
SetColumnTypeTask ..|> DDLDefinitionTask_SetColumnType
SetColumnTypeTask --> Metadata : uses
SetColumnTypeTask --> AccessControl : uses
SetColumnTypeTask --> Session : uses
SetColumnTypeTask --> SetColumnType : executes
SetColumnTypeTask --> TableHandle : resolves
SetColumnTypeTask --> ColumnHandle : resolves
SetColumnTypeTask --> Type : parses
Metadata <.. PrestoDataDefBindingHelper : referenced
TypeSignature <.. SetColumnTypeTask : parseTypeSignature
class DDLDefinitionTask_SetColumnType {
<<interface>>
+String getName()
+ListenableFuture execute(SetColumnType statement, TransactionManager transactionManager, Metadata metadata, AccessControl accessControl, Session session, List parameters, WarningCollector warningCollector, String query)
}
Class diagram for connector metadata setColumnType implementationsclassDiagram
class Metadata {
+void setColumnType(Session session, TableHandle tableHandle, ColumnHandle column, Type type)
}
class MetadataManager {
+void setColumnType(Session session, TableHandle tableHandle, ColumnHandle column, Type type)
}
class MetadataManagerStats {
-AtomicLong setColumnTypeCalls
-TimeStat setColumnTypeTime
+void recordSetColumnTypeCall(long duration)
}
class StatsRecordingMetadataManager {
-Metadata delegate
-MetadataManagerStats stats
+void setColumnType(Session session, TableHandle tableHandle, ColumnHandle column, Type type)
}
class ConnectorMetadata {
+void setColumnType(ConnectorSession session, ConnectorTableHandle tableHandle, ColumnHandle column, Type type)
}
class ClassLoaderSafeConnectorMetadata {
-ConnectorMetadata delegate
-ClassLoader classLoader
+void setColumnType(ConnectorSession session, ConnectorTableHandle tableHandle, ColumnHandle columnHandle, Type type)
}
class BlackHoleMetadata {
-Map tables
+void setColumnType(ConnectorSession session, ConnectorTableHandle tableHandle, ColumnHandle columnHandle, Type type)
}
class BlackHoleTableHandle {
+List getColumnHandles()
+String getSchemaName()
+String getTableName()
+int getSplitCount()
+int getPagesPerSplit()
+int getRowsPerPage()
+int getFieldsLength()
+Duration getPageProcessingDelay()
}
class BlackHoleColumnHandle {
+String getName()
}
class IcebergAbstractMetadata {
+void setColumnType(ConnectorSession session, ConnectorTableHandle tableHandle, ColumnHandle columnHandle, Type type)
}
class IcebergTableHandle {
+SchemaTableName getSchemaTableName()
}
class TestingMetadata {
+void setColumnType(ConnectorSession session, ConnectorTableHandle tableHandle, ColumnHandle column, Type type)
}
class RemoteMetadataManager {
+void setColumnType(Session session, TableHandle tableHandle, ColumnHandle column, Type type)
}
class ConnectorSession {
}
class TableHandle {
+ConnectorId getConnectorId()
+ConnectorTableHandle getConnectorHandle()
}
class ConnectorTableHandle {
}
class ColumnHandle {
}
class Session {
+ConnectorSession toConnectorSession(ConnectorId connectorId)
}
class ConnectorId {
}
class Type {
}
class Table {
+UpdateSchema updateSchema()
}
class UpdateSchema {
+UpdateSchema updateColumn(String name, Type type)
+void commit()
}
class IcebergErrorCode {
+ICEBERG_INCOMPATIBLE_COLUMN_TYPE
}
class PrestoException {
}
class AtomicLong {
}
class TimeStat {
+void add(long duration, TimeUnit unit)
}
class TimeUnit {
}
Metadata <|.. MetadataManager
MetadataManager --> ConnectorMetadata : getMetadataForWrite
MetadataManager --> MetadataManagerStats : uses
MetadataManagerStats <.. StatsRecordingMetadataManager
Metadata <|.. StatsRecordingMetadataManager
StatsRecordingMetadataManager --> Metadata : delegate
ConnectorMetadata <|.. BlackHoleMetadata
ConnectorMetadata <|.. IcebergAbstractMetadata
ConnectorMetadata <|.. TestingMetadata
MetadataManager --> ConnectorMetadata : setColumnType
ClassLoaderSafeConnectorMetadata --> ConnectorMetadata : delegate
BlackHoleMetadata --> BlackHoleTableHandle : casts
BlackHoleMetadata --> BlackHoleColumnHandle : uses
IcebergAbstractMetadata --> IcebergTableHandle : casts
IcebergAbstractMetadata --> Table : getIcebergTable
IcebergAbstractMetadata --> IcebergErrorCode : uses
IcebergAbstractMetadata --> PrestoException : throws
TestingMetadata --> ConnectorTableMetadata : uses
RemoteMetadataManager ..|> Metadata
class ConnectorTableMetadata {
+SchemaTableName getTable()
+List getColumns()
+Map getProperties()
+Optional getComment()
}
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Motivation and Context
Impact
Test Plan
Contributor checklist
Release Notes
Please follow release notes guidelines and fill in the release notes below.
If release note is NOT required, use:
Summary by Sourcery
Add support for ALTER TABLE ... ALTER COLUMN SET DATA TYPE, including metadata plumbing, access control, Iceberg and Blackhole connector implementations, and associated statistics and error handling.
New Features:
Bug Fixes:
Enhancements:
Documentation:
Tests: