Skip to content

Conversation

Lee-W
Copy link
Contributor

@Lee-W Lee-W commented May 9, 2025

Summary

  • Remove the following rules
    • name
      • airflow.auth.managers.base_auth_manager.is_authorized_datasetairflow.api_fastapi.auth.managers.base_auth_manager.is_authorized_asset
      • airflow.providers.fab.auth_manager.fab_auth_manager.is_authorized_datasetairflow.providers.fab.auth_manager.fab_auth_manager.is_authorized_asset
  • Update the following rules
    • name
      • airflow.models.baseoperatorlink.BaseOperatorLinkairflow.sdk.BaseOperatorLink
      • airflow.api_connexion.security.requires_access → "Use airflow.api_fastapi.core_api.security.requires_access_* instead`"
      • airflow.api_connexion.security.requires_access_datasetairflow.api_fastapi.core_api.security.requires_access_asset
      • airflow.notifications.basenotifier.BaseNotifierairflow.sdk.bases.notifier.BaseNotifier
      • airflow.www.auth.has_access → None
      • airflow.www.auth.has_access_dataset → None
      • airflow.www.utils.get_sensitive_variables_fields→ None
      • airflow.www.utils.should_hide_value_for_key→ None
    • class attribute
      • airflow..sensors.weekday.DayOfWeekSensor
        • use_task_execution_day removed
      • airflow.providers.amazon.aws.auth_manager.aws_auth_manager.AwsAuthManager
        • is_authorized_dataset
  • Add the following rules
    • class attribute
      • airflow.auth.managers.base_auth_manager.BaseAuthManager | airflow.providers.fab.auth_manager.fab_auth_manager.FabAuthManager
    • name
      • airflow.auth.managers.base_auth_manager.BaseAuthManagerairflow.api_fastapi.auth.managers.base_auth_manager.BaseAuthManager * is_authorized_datasetis_authorized_asset
  • refactor
    • simplify unnecessary match with if else
    • rename Replacement::Name as Replacement::AttrName

Test Plan

The test fixtures have been revised and updated.

@Lee-W Lee-W mentioned this pull request May 9, 2025
2 tasks
@Lee-W Lee-W changed the title Update air301 air311 rules [airflow] Update AIR301, AIR311 rules with the latest Airflow implemnetation May 9, 2025
@Lee-W Lee-W force-pushed the update-AIR301-AIR311-rules branch 2 times, most recently from dedaab9 to 4e3f5ef Compare May 12, 2025 15:00
Copy link
Contributor

github-actions bot commented May 12, 2025

ruff-ecosystem results

Linter (stable)

✅ ecosystem check detected no linter changes.

Linter (preview)

ℹ️ ecosystem check detected linter changes. (+21 -2 violations, +0 -0 fixes in 1 projects; 54 projects unchanged)

apache/airflow (+21 -2 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --no-fix --output-format concise --preview --select ALL

+ dev/airflow_perf/scheduler_dag_execution_timing.py:255:10: AIR301 `airflow.utils.db.create_session` is removed in Airflow 3.0
- dev/airflow_perf/scheduler_dag_execution_timing.py:255:13: AIR301 `airflow.utils.db.create_session` is removed in Airflow 3.0
+ dev/airflow_perf/scheduler_dag_execution_timing.py:302:18: AIR301 `airflow.utils.db.create_session` is removed in Airflow 3.0
- dev/airflow_perf/scheduler_dag_execution_timing.py:302:21: AIR301 `airflow.utils.db.create_session` is removed in Airflow 3.0
+ providers/amazon/src/airflow/providers/amazon/aws/links/base_aws.py:41:19: AIR311 `airflow.models.baseoperatorlink.BaseOperatorLink` is removed in Airflow 3.0; It still works in Airflow 3.0 but is expected to be removed in a future version.
+ providers/databricks/src/airflow/providers/databricks/operators/databricks.py:250:28: AIR311 `airflow.models.baseoperatorlink.BaseOperatorLink` is removed in Airflow 3.0; It still works in Airflow 3.0 but is expected to be removed in a future version.
+ providers/databricks/src/airflow/providers/databricks/plugins/databricks_workflow.py:238:26: AIR311 `airflow.models.baseoperatorlink.BaseOperatorLink` is removed in Airflow 3.0; It still works in Airflow 3.0 but is expected to be removed in a future version.
+ providers/databricks/src/airflow/providers/databricks/plugins/databricks_workflow.py:272:38: AIR311 `airflow.models.baseoperatorlink.BaseOperatorLink` is removed in Airflow 3.0; It still works in Airflow 3.0 but is expected to be removed in a future version.
+ providers/databricks/src/airflow/providers/databricks/plugins/databricks_workflow.py:366:39: AIR311 `airflow.models.baseoperatorlink.BaseOperatorLink` is removed in Airflow 3.0; It still works in Airflow 3.0 but is expected to be removed in a future version.
+ providers/dbt/cloud/src/airflow/providers/dbt/cloud/operators/dbt.py:50:34: AIR311 `airflow.models.baseoperatorlink.BaseOperatorLink` is removed in Airflow 3.0; It still works in Airflow 3.0 but is expected to be removed in a future version.
+ providers/google/src/airflow/providers/google/cloud/links/base.py:38:22: AIR311 `airflow.models.baseoperatorlink.BaseOperatorLink` is removed in Airflow 3.0; It still works in Airflow 3.0 but is expected to be removed in a future version.
+ providers/google/src/airflow/providers/google/cloud/links/datafusion.py:44:22: AIR311 `airflow.models.baseoperatorlink.BaseOperatorLink` is removed in Airflow 3.0; It still works in Airflow 3.0 but is expected to be removed in a future version.
+ providers/google/src/airflow/providers/google/cloud/links/dataproc.py:136:24: AIR311 `airflow.models.baseoperatorlink.BaseOperatorLink` is removed in Airflow 3.0; It still works in Airflow 3.0 but is expected to be removed in a future version.
+ providers/google/src/airflow/providers/google/cloud/links/dataproc.py:83:20: AIR311 `airflow.models.baseoperatorlink.BaseOperatorLink` is removed in Airflow 3.0; It still works in Airflow 3.0 but is expected to be removed in a future version.
+ providers/google/src/airflow/providers/google/cloud/operators/dataproc_metastore.py:112:37: AIR311 `airflow.models.baseoperatorlink.BaseOperatorLink` is removed in Airflow 3.0; It still works in Airflow 3.0 but is expected to be removed in a future version.
+ providers/google/src/airflow/providers/google/cloud/operators/dataproc_metastore.py:64:29: AIR311 `airflow.models.baseoperatorlink.BaseOperatorLink` is removed in Airflow 3.0; It still works in Airflow 3.0 but is expected to be removed in a future version.
+ providers/google/src/airflow/providers/google/marketing_platform/links/analytics_admin.py:39:31: AIR311 `airflow.models.baseoperatorlink.BaseOperatorLink` is removed in Airflow 3.0; It still works in Airflow 3.0 but is expected to be removed in a future version.
+ providers/microsoft/azure/src/airflow/providers/microsoft/azure/operators/data_factory.py:52:53: AIR311 `airflow.models.baseoperatorlink.BaseOperatorLink` is removed in Airflow 3.0; It still works in Airflow 3.0 but is expected to be removed in a future version.
+ providers/microsoft/azure/src/airflow/providers/microsoft/azure/operators/powerbi.py:46:19: AIR311 `airflow.models.baseoperatorlink.BaseOperatorLink` is removed in Airflow 3.0; It still works in Airflow 3.0 but is expected to be removed in a future version.
+ providers/microsoft/azure/src/airflow/providers/microsoft/azure/operators/synapse.py:133:35: AIR311 `airflow.models.baseoperatorlink.BaseOperatorLink` is removed in Airflow 3.0; It still works in Airflow 3.0 but is expected to be removed in a future version.
+ providers/standard/src/airflow/providers/standard/operators/trigger_dagrun.py:81:25: AIR311 `airflow.models.baseoperatorlink.BaseOperatorLink` is removed in Airflow 3.0; It still works in Airflow 3.0 but is expected to be removed in a future version.
+ providers/standard/src/airflow/providers/standard/sensors/external_task.py:72:23: AIR311 `airflow.models.baseoperatorlink.BaseOperatorLink` is removed in Airflow 3.0; It still works in Airflow 3.0 but is expected to be removed in a future version.
+ providers/yandex/src/airflow/providers/yandex/links/yq.py:43:14: AIR311 `airflow.models.baseoperatorlink.BaseOperatorLink` is removed in Airflow 3.0; It still works in Airflow 3.0 but is expected to be removed in a future version.

Changes by rule (2 rules affected)

code total + violation - violation + fix - fix
AIR311 19 19 0 0 0
AIR301 4 2 2 0 0

@ntBre ntBre self-requested a review May 13, 2025 01:40
@Lee-W Lee-W force-pushed the update-AIR301-AIR311-rules branch from 7c2b1e3 to bbaae8e Compare May 15, 2025 03:13
@Lee-W Lee-W marked this pull request as ready for review May 15, 2025 08:44
@Lee-W Lee-W force-pushed the update-AIR301-AIR311-rules branch from bbaae8e to a455884 Compare May 15, 2025 10:35
@ntBre ntBre added rule Implementing or modifying a lint rule preview Related to preview mode features labels May 15, 2025
Copy link
Contributor

@ntBre ntBre left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This overall looks good to me. I had a few minor suggestions and one question about the import code.

diagnostic.try_set_fix(|| {
let (import_edit, binding) = checker.importer().get_or_import_symbol(
&ImportRequest::import_from(module, name),
let (import_edit, _) = checker.importer().get_or_import_symbol(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you're supposed to use the binding returned by get_or_import_symbol. Was something going wrong here when you did that?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There 2 cases like the following that need to be handled this way.

from airflow.providers.amazon.aws.auth_manager.avp.entities import AvpEntities

AvpEntities.DATASET

which should be fixed as

from airflow.providers.amazon.aws.auth_manager.avp.entities import AvpEntities

AvpEntities.ASSET

is we use binding, it will be fixed as

from airflow.providers.amazon.aws.auth_manager.avp.entities import AvpEntities

AvpEntities

which is wrong

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, I think that makes sense. Thanks!

@Lee-W Lee-W force-pushed the update-AIR301-AIR311-rules branch 3 times, most recently from f61f06c to d75296e Compare May 16, 2025 13:42
Lee-W added 8 commits May 19, 2025 23:08
* Remove the following rules
    * name
        * `airflow.auth.managers.base_auth_manager.is_authorized_dataset` → `airflow.api_fastapi.auth.managers.base_auth_manager.is_authorized_asset`
        * `airflow.providers.fab.auth_manager.fab_auth_manager.is_authorized_dataset` → `airflow.providers.fab.auth_manager.fab_auth_manager.is_authorized_asset`
* Update the following rules
    * name
        * `airflow.api_connexion.security.requires_access` → "Use `airflow.api_fastapi.core_api.security.requires_access_*` instead`"
        * `airflow.api_connexion.security.requires_access_dataset`→ `airflow.api_fastapi.core_api.security.requires_access_asset`
        * `airflow.notifications.basenotifier.BaseNotifier` → `airflow.sdk.bases.notifier.BaseNotifier`
        * `airflow.www.auth.has_access`  → None
        * `airflow.www.auth.has_access_dataset` → None
        * `airflow.www.utils.get_sensitive_variables_fields`→ None
        * `airflow.www.utils.should_hide_value_for_key`→ None
* Add the following rules
    * class attribute
        * `airflow.auth.managers.base_auth_manager.BaseAuthManager` | `airflow.providers.fab.auth_manager.fab_auth_manager.FabAuthManager`
            * `is_authorized_dataset` → `is_authorized_asset`
* Update the following rules
    * class attribute
        * `airflow..sensors.weekday.DayOfWeekSensor`
            * `use_task_execution_day` removed
        * `airflow.providers.amazon.aws.auth_manager.aws_auth_manager.AwsAuthManager`
            * `is_authorized_dataset`
* Add the following rules
    * name
        * `airflow.auth.managers.base_auth_manager.BaseAuthManager` → `airflow.api_fastapi.auth.managers.base_auth_manager.BaseAuthManager`
@Lee-W Lee-W force-pushed the update-AIR301-AIR311-rules branch from d75296e to d631292 Compare May 19, 2025 15:08
@ntBre ntBre changed the title [airflow] Update AIR301, AIR311 rules with the latest Airflow implemnetation [airflow] Update AIR301 and AIR311 with the latest Airflow implementations May 19, 2025
@ntBre ntBre merged commit 236633c into astral-sh:main May 19, 2025
34 checks passed
dcreager added a commit that referenced this pull request May 19, 2025
…rals

* origin/main:
  [ty] Add hint that PEP 604 union syntax is only available in 3.10+ (#18192)
  Unify `Message` variants (#18051)
  [`airflow`] Update `AIR301` and `AIR311` with the latest Airflow implementations (#17985)
  [`airflow`] Move rules from `AIR312` to `AIR302` (#17940)
  [ty] Small LSP cleanups (#18201)
  [ty] Show related information in diagnostic (#17359)
  Default `src.root` to `['.', '<project_name>']` if the directory exists (#18141)
dcreager added a commit that referenced this pull request May 19, 2025
* main:
  [ty] Use first matching constructor overload when inferring specializations (#18204)
  [ty] Add hint that PEP 604 union syntax is only available in 3.10+ (#18192)
  Unify `Message` variants (#18051)
  [`airflow`] Update `AIR301` and `AIR311` with the latest Airflow implementations (#17985)
  [`airflow`] Move rules from `AIR312` to `AIR302` (#17940)
  [ty] Small LSP cleanups (#18201)
  [ty] Show related information in diagnostic (#17359)
  Default `src.root` to `['.', '<project_name>']` if the directory exists (#18141)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
preview Related to preview mode features rule Implementing or modifying a lint rule
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants