Skip to content

[SPARK-29290][CORE] Update to chill 0.9.5 #27227

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from

Conversation

srowen
Copy link
Member

@srowen srowen commented Jan 16, 2020

What changes were proposed in this pull request?

Update Twitter Chill to 0.9.5.

Why are the changes needed?

Primarily, Scala 2.13 support for later.
Other changes from 0.9.3 are apparently just minor fixes and improvements:
https://github.com/twitter/chill/releases

Does this PR introduce any user-facing change?

No

How was this patch tested?

Existing tests

@srowen srowen self-assigned this Jan 16, 2020
@SparkQA
Copy link

SparkQA commented Jan 16, 2020

Test build #116813 has finished for PR 27227 at commit 89e2af4.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Jan 16, 2020

Test build #116811 has finished for PR 27227 at commit dbc819a.

  • This patch passes all tests.
  • This patch does not merge cleanly.
  • This patch adds no public classes.

@dongjoon-hyun
Copy link
Member

Retest this please.

@SparkQA
Copy link

SparkQA commented Jan 17, 2020

Test build #116879 has finished for PR 27227 at commit 89e2af4.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@dongjoon-hyun
Copy link
Member

dongjoon-hyun commented Jan 17, 2020

The patch itself looks good to me, @srowen . The library is published yesterday and it seems that Google Maven Central needs more time to pick up that. Shall we wait for one or two days?

[ERROR] Failed to execute goal on project spark-unsafe_2.12:
Could not resolve dependencies for project org.apache.spark:spark-unsafe_2.12:jar:3.0.0-SNAPSHOT: 
Could not find artifact com.twitter:chill_2.12:jar:0.9.5 in google-maven-central 
(https://maven-central.storage-download.googleapis.com/repos/central/data/) -> [Help 1]

@srowen
Copy link
Member Author

srowen commented Jan 18, 2020

Jenkins, retest this please

@SparkQA
Copy link

SparkQA commented Jan 18, 2020

Test build #116978 has finished for PR 27227 at commit 89e2af4.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@srowen
Copy link
Member Author

srowen commented Jan 18, 2020

@dongjoon-hyun yeah I noticed this locally when using the Google mirror. Hm, I'm not sure how frequently they sync; this could become an issue. This isn't urgent so I'll sit on it a while, but after a week or so maybe we have to merge and then disable the mirror

@dongjoon-hyun
Copy link
Member

dongjoon-hyun commented Jan 18, 2020

@srowen . If we disable the mirror, Maven Central will cause a flaky failure in GitHub Action. I'm also searching a nice way to avoid this sync issue. For now, there was no luck to me.

@dongjoon-hyun
Copy link
Member

dongjoon-hyun commented Jan 18, 2020

BTW, Maven Central also looks weird to me. Technically, we can access https://repo1.maven.org/maven2/com/twitter/chill_2.12/0.9.5/ correctly, but the listing doesn't show it.

Screen Shot 2020-01-18 at 3 09 40 PM

However, the maven-metadata.xml is correct.

@srowen
Copy link
Member Author

srowen commented Jan 18, 2020

For the latter, try refreshing. I do see it and can see https://repo1.maven.org/maven2/com/twitter/chill_2.12/0.9.5/

@dongjoon-hyun
Copy link
Member

No. What I mean is https://repo1.maven.org/maven2/com/twitter/chill_2.12/.

@dongjoon-hyun
Copy link
Member

You can not see 0.9.5 in the upper directory. (I've been using Private Browsing mode always to check this location). Only maven-metadata.xml is correct.

@srowen
Copy link
Member Author

srowen commented Jan 18, 2020

I do see 0.9.5 in the parent directory:
image

@dongjoon-hyun
Copy link
Member

Oh, that's weird. I'm using both Chrome (Incognito) and Safari (Private) mode, but I cannot see it still (as of now). Is this regional ISP (Internet Service Provider) issue? Hmm.

@dongjoon-hyun
Copy link
Member

The following is curl result.

$ curl https://repo1.maven.org/maven2/com/twitter/chill_2.12/ | grep 0.9
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  2255  100  2255    0     0  15445      0 --:--:-- --:--:-- --:--:-- 15445
<a href="0.9.0/" title="0.9.0/">0.9.0/</a>                                            2017-01-03 19:41         -
<a href="0.9.1/" title="0.9.1/">0.9.1/</a>                                            2017-01-09 18:24         -
<a href="0.9.2/" title="0.9.2/">0.9.2/</a>                                            2017-02-23 19:16         -
<a href="0.9.3/" title="0.9.3/">0.9.3/</a>                                            2018-08-11 21:55         -
<a href="0.9.4/" title="0.9.4/">0.9.4/</a>                                            2019-11-19 02:09         -
<a href="maven-metadata.xml" title="maven-metadata.xml">maven-metadata.xml</a>                                2019-11-19 02:15       669
<a href="maven-metadata.xml.md5" title="maven-metadata.xml.md5">maven-metadata.xml.md5</a>                            2019-11-19 02:15        32
<a href="maven-metadata.xml.sha1" title="maven-metadata.xml.sha1">maven-metadata.xml.sha1</a>                           2019-11-19 02:15        40

@srowen
Copy link
Member Author

srowen commented Jan 18, 2020

Weird, I curl'ed it too and do see 0.9.5. We may indeed be hitting different instances of the central repo. That is itself a bit troublesome if not all of even central has the artifact

@dongjoon-hyun
Copy link
Member

Hi, @srowen . I found a solution for this. Could you review this?

Copy link
Member

@dongjoon-hyun dongjoon-hyun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, LGTM. Merged to master.
Thank you for waiting, @srowen .

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants