[MINOR][TESTS] Ignore GitHub Action and AppVeyor file changes in testing #26556

dongjoon-hyun · 2019-11-16T09:01:21Z

What changes were proposed in this pull request?

This PR aims to ignore GitHub Action and AppVeyor file changes. When we touch these files, Jenkins job should not trigger a full testing.

Why are the changes needed?

Currently, these files are categorized to root and trigger the full testing and ends up wasting the Jenkins resources.

[SPARK-29378][R] Upgrade SparkR to use Arrow 0.15 API #26555

[info] Using build tool sbt with Hadoop profile hadoop2.7 under environment amplab_jenkins
From https://github.com/apache/spark
 * [new branch]      master     -> master
[info] Found the following changed modules: sparkr, root
[info] Setup the following environment variables for tests:

Does this PR introduce any user-facing change?

No. (Jenkins testing only).

How was this patch tested?

Manually.

$ dev/run-tests.py -h -v
...
Trying:
    [x.name for x in determine_modules_for_files([".github/workflows/master.yml", "appveyor.xml"])]
Expecting:
    []
...

SparkQA · 2019-11-16T09:07:42Z

Test build #113919 has finished for PR 26556 at commit 70fa59b.

This patch fails Python style tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2019-11-16T11:54:36Z

Test build #113920 has finished for PR 26556 at commit f64d060.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

dongjoon-hyun · 2019-11-16T16:49:09Z

Hi, @srowen and @HyukjinKwon .
This is helpful when we investigate AppVeyor or GitHub Action only issue.
This will reduce the waste of our Jenkins resources.
Could you review this PR?

dongjoon-hyun · 2019-11-16T17:24:41Z

As I mentioned in #26133 , AppVeyor is the only way to run SparkR arrow test. I'll merge this because we need more job triggers to upgrade SparkR Arrow.

dongjoon-hyun · 2019-11-16T17:25:14Z

Merged to master.

HyukjinKwon · 2019-11-18T03:59:47Z

dev/run-tests.py


    >>> sorted(x.name for x in determine_modules_for_files(["python/pyspark/a.py", "sql/core/foo"]))
    ['pyspark-core', 'sql']
    >>> [x.name for x in determine_modules_for_files(["file_not_matched_by_any_subproject"])]
    ['root']
+    >>> [x.name for x in determine_modules_for_files( \
+            [".github/workflows/master.yml", "appveyor.yml"])]


no big deal but it is usually:

>>> [x.name for x in determine_modules_for_files( ... [".github/workflows/master.yml", "appveyor.yml"])]

HyukjinKwon

Looks good to me.

### What changes were proposed in this pull request? This PR aims to ignore `GitHub Action` and `AppVeyor` file changes. When we touch these files, Jenkins job should not trigger a full testing. ### Why are the changes needed? Currently, these files are categorized to `root` and trigger the full testing and ends up wasting the Jenkins resources. - #26555 ``` [info] Using build tool sbt with Hadoop profile hadoop2.7 under environment amplab_jenkins From https://github.com/apache/spark * [new branch] master -> master [info] Found the following changed modules: sparkr, root [info] Setup the following environment variables for tests: ``` ### Does this PR introduce any user-facing change? No. (Jenkins testing only). ### How was this patch tested? Manually. ``` $ dev/run-tests.py -h -v ... Trying: [x.name for x in determine_modules_for_files([".github/workflows/master.yml", "appveyor.xml"])] Expecting: [] ... ``` Closes #26556 from dongjoon-hyun/SPARK-IGNORE-APPVEYOR. Authored-by: Dongjoon Hyun <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]> (cherry picked from commit d0470d6) Signed-off-by: Dongjoon Hyun <[email protected]>

…ting ### What changes were proposed in this pull request? #26556 excluded `.github/workflows/master.yml`. So tests are skipped if the GitHub Actions configuration file is changed. As of SPARK-32245, we now run the regular tests via the testing script. We should include it to test to make sure GitHub Actions build does not break due to some changes such as Python versions. ### Why are the changes needed? For better test coverage in GitHub Actions build. ### Does this PR introduce _any_ user-facing change? No, dev-only. ### How was this patch tested? GitHub Actions in this PR will test. Closes #29305 from HyukjinKwon/SPARK-32496. Authored-by: HyukjinKwon <[email protected]> Signed-off-by: HyukjinKwon <[email protected]>

…ting ### What changes were proposed in this pull request? apache#26556 excluded `.github/workflows/master.yml`. So tests are skipped if the GitHub Actions configuration file is changed. As of SPARK-32245, we now run the regular tests via the testing script. We should include it to test to make sure GitHub Actions build does not break due to some changes such as Python versions. ### Why are the changes needed? For better test coverage in GitHub Actions build. ### Does this PR introduce _any_ user-facing change? No, dev-only. ### How was this patch tested? GitHub Actions in this PR will test. Closes apache#29305 from HyukjinKwon/SPARK-32496. Authored-by: HyukjinKwon <[email protected]> Signed-off-by: HyukjinKwon <[email protected]>

[MINOR][TESTS] Ignore GitHub Action and AppVeyor files

70fa59b

dongjoon-hyun added the TESTS label Nov 16, 2019

dongjoon-hyun changed the title ~~[MINOR][TESTS] Ignore GitHub Action and AppVeyor files~~ [MINOR][TESTS] Ignore GitHub Action and AppVeyor files in testing Nov 16, 2019

dongjoon-hyun changed the title ~~[MINOR][TESTS] Ignore GitHub Action and AppVeyor files in testing~~ [MINOR][TESTS] Ignore GitHub Action and AppVeyor file changes in testing Nov 16, 2019

fix appveyor.yml

0a55273

fix indentation

f64d060

dongjoon-hyun closed this in d0470d6 Nov 16, 2019

dongjoon-hyun deleted the SPARK-IGNORE-APPVEYOR branch November 16, 2019 17:26

HyukjinKwon reviewed Nov 18, 2019

View reviewed changes

HyukjinKwon mentioned this pull request Jul 30, 2020

[SPARK-32496][INFRA] Include GitHub Action file as the changes in testing #29305

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[MINOR][TESTS] Ignore GitHub Action and AppVeyor file changes in testing #26556

[MINOR][TESTS] Ignore GitHub Action and AppVeyor file changes in testing #26556

Uh oh!

dongjoon-hyun commented Nov 16, 2019 •

edited

Loading

Uh oh!

SparkQA commented Nov 16, 2019

Uh oh!

SparkQA commented Nov 16, 2019

Uh oh!

dongjoon-hyun commented Nov 16, 2019

Uh oh!

dongjoon-hyun commented Nov 16, 2019

Uh oh!

dongjoon-hyun commented Nov 16, 2019

Uh oh!

HyukjinKwon Nov 18, 2019

Uh oh!

HyukjinKwon left a comment

Uh oh!

Uh oh!

[MINOR][TESTS] Ignore GitHub Action and AppVeyor file changes in testing #26556

[MINOR][TESTS] Ignore GitHub Action and AppVeyor file changes in testing #26556

Uh oh!

Conversation

dongjoon-hyun commented Nov 16, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changes were proposed in this pull request?

Why are the changes needed?

Does this PR introduce any user-facing change?

How was this patch tested?

Uh oh!

SparkQA commented Nov 16, 2019

Uh oh!

SparkQA commented Nov 16, 2019

Uh oh!

dongjoon-hyun commented Nov 16, 2019

Uh oh!

dongjoon-hyun commented Nov 16, 2019

Uh oh!

dongjoon-hyun commented Nov 16, 2019

Uh oh!

HyukjinKwon Nov 18, 2019

Choose a reason for hiding this comment

Uh oh!

HyukjinKwon left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

dongjoon-hyun commented Nov 16, 2019 •

edited

Loading