Skip to content

Releases: aws-samples/dbt-glue

1.10.15

26 Nov 09:39
158426a

Choose a tag to compare

  • Added ability to have specific models use the same glue session separate from the project level session by specifying a "group_session_id" in the model's meta.
  • Ensured that when "group_session_id" is set, "enable_session_per_model" is not enabled for that model so that session usage behavior acts appropriately.
  • Added testing to capture proper behavior of "group_session_id".
  • Upgrade dependencies: dbt-core 1.10.15, moto 5.1.17, black 25.11.0.

1.10.13

15 Oct 16:24
c8cab57

Choose a tag to compare

  • Added ability to use Spark temporary views when using iceberg targets instead of physical tables to store intermediate results, by setting model config use_iceberg_temp_views to 'True'
  • Added ability to purge iceberg and s3tables underlying data s3 object during drop table when config purge_dropped_iceberg_data is set to 'True' in model config
  • Improved unit testing of drop_relation by separating out glue__get_drop_sql and unit testing the output for each condition
  • Improved coverage of unit testing for glue__make_target_relation and glue__create_temporary_view
  • Separated glue__make_target_relation into utils/make_target_relation.sql so that we could mock this often called macro, allowing us to better unit test macros that call this one
  • Fixed several bugs on S3 tables support

1.10.11

19 Sep 01:15
79ba9c4

Choose a tag to compare

  • Fix duplicate table macro issue in published package (GitHub #608)
  • Add test to prevent duplicate macro definitions in future releases
  • Improve build process to ensure clean package builds
  • Fix issue where including unrelated values in meta section would create new sessions when glue_session_id is set
  • Fix NoneType error in meta configuration handling
  • Upgrade dependencies: dbt-core 1.10.11, dbt-spark 1.9.3, moto 5.1.12, mypy 1.18.1.

1.10.9

20 Aug 09:27
90727b1

Choose a tag to compare

  • Allow temporary tables to be created in different schema/database than target model using profile temp-schema variable
  • Add experimental Python model support with Iceberg file format (requires AWS Glue 4.0+)
  • Add experimental Amazon S3 Tables support with file_format='s3tables'
  • Upgrade dependencies: dbt-core 1.10.9, dbt-tests-adapter 1.18.0, moto 5.1.10, freezegun 1.5.5.
  • Fix merge strategy with schema changes for Iceberg tables

1.9.4

14 Apr 11:07
27a777d

Choose a tag to compare

  • Reduce debug logging
  • Fix invalid Spark relation type: iceberg_table
  • Fix Iceberg snapshot issue of incorrect DROP VIEW calls
  • Upgrade dependencies: dbt-core 1.9.4, dbt-spark 1.9.2, moto 5.1.3.

1.9.2

24 Mar 05:50
7ed4830

Choose a tag to compare

  • Correctly handle EntityNotFound when trying to determine session state, setting state to does not exist instead of STOPPED.
  • Allow spawning new isolated sessions for the models that require different session configuration.
  • Correctly handle EntityNotFound when listing relations.
  • Add configuration property to allow spark casting of seed column types.
  • Fix the get_columns_in_relation function error when on_schema_change is specified.
  • Upgrade dependencies: dbt-core 1.9.2, dbt-spark 1.9.1, dbt-tests-adapter 1.11.0, mypy 1.15.0, moto 5.0.28, and black 25.1.0.
  • Fix the temporary view creation issue when using schema_on_change
  • Fix error handling and logging
  • Fix missing catalog identifier related issues for Iceberg
  • Add update_iceberg_ts column with an add_iceberg_timestamp option.
  • Add refresh table when missing columns are detected
  • Fix ALTER TABLE ADD COLUMN returning None

1.9.0

12 Dec 08:54
d8259f9

Choose a tag to compare

  • Allow to load big seed files
  • Migrates the PySpark code for the Iceberg file format at a macro level, making the impl.py file more readable.
  • Fixes the get_columns_in_relation function to work for both Iceberg and non-Iceberg tables without hard-coding the catalog name.
  • Fixes the get_location table function to work for both Iceberg and non-Iceberg tables on macOS and Windows.
  • Adds a helper function to retrieve the Iceberg catalog namespace from the profile.yaml file.
  • Adds merge_exclude_columns and incremental_predicates features.
  • Drop Python 3.8 support
  • Upgrade default Glue version to 5.0
  • Upgrade dependencies: dbt-core 1.9.0 and dbt-spark 1.9.0
    https://github.com/aws-samples/dbt-glue/blob/main/CHANGELOG.md

1.8.6

26 Sep 04:28
7c00efa

Choose a tag to compare

  • Fix session provisioning timeout and delay handling
  • Add write options for Delta format
  • Add on_schema_change possibility
  • Fix table materialization for Delta models
  • Change GlueColumn parent from base Column to SparkColumn
  • Fix unraised DbtDatabaseError
  • Fix get_columns_in_relation function to stop returning additional partition columns
  • Fix null values handling in seeds
  • Fix exceptions import for FailedToConnectError and ExecutableError
  • Fix the case-sensitive comparison on the seed name

https://github.com/aws-samples/dbt-glue/blob/main/CHANGELOG.md