Skip to content

[SPARK-2936] Migrate Netty network module from Java to Scala #1865

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 3 commits into from
Closed

[SPARK-2936] Migrate Netty network module from Java to Scala #1865

wants to merge 3 commits into from

Conversation

rxin
Copy link
Contributor

@rxin rxin commented Aug 9, 2014

The Netty network module was originally written when Scala 2.9.x had a bug that prevents a pure Scala implementation, and a subset of the files were done in Java. We have since upgraded to Scala 2.10, and can migrate all Java files now to Scala.

netty/netty#781

mesos/spark#522

@SparkQA
Copy link

SparkQA commented Aug 9, 2014

QA tests have started for PR 1865. This patch merges cleanly.
View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18242/consoleFull

@SparkQA
Copy link

SparkQA commented Aug 9, 2014

QA results for PR 1865:
- This patch PASSES unit tests.
- This patch merges cleanly
- This patch adds no public classes

For more information see test ouptut:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18242/consoleFull

@SparkQA
Copy link

SparkQA commented Aug 9, 2014

QA tests have started for PR 1865. This patch merges cleanly.
View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18250/consoleFull

@SparkQA
Copy link

SparkQA commented Aug 9, 2014

QA results for PR 1865:
- This patch FAILED unit tests.
- This patch merges cleanly
- This patch adds no public classes

For more information see test ouptut:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18250/consoleFull

@rxin
Copy link
Contributor Author

rxin commented Aug 9, 2014

Jenkins, retest this please.

@SparkQA
Copy link

SparkQA commented Aug 9, 2014

QA tests have started for PR 1865. This patch merges cleanly.
View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18256/consoleFull

@SparkQA
Copy link

SparkQA commented Aug 9, 2014

QA results for PR 1865:
- This patch PASSES unit tests.
- This patch merges cleanly
- This patch adds no public classes

For more information see test ouptut:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18256/consoleFull

aarondav
aarondav reviewed Aug 10, 2014
View reviewed changes
core/src/main/scala/org/apache/spark/network/netty/FileServerHandler.scala
extends SimpleChannelInboundHandler[String] with Logging {

override def channelRead0(ctx: ChannelHandlerContext, blockIdString: String): Unit = {
val blockId: BlockId = BlockId.apply(blockIdString)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BlockId(blockIdString)

@aarondav
Copy link
Contributor

Just had a couple minor issues with the translation, LGTM functionality-wise. Did not do a thorough diff check, though.

@SparkQA
Copy link

SparkQA commented Aug 11, 2014

QA tests have started for PR 1865. This patch merges cleanly.
View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18292/consoleFull

@SparkQA
Copy link

SparkQA commented Aug 11, 2014

QA results for PR 1865:
- This patch PASSES unit tests.
- This patch merges cleanly
- This patch adds no public classes

For more information see test ouptut:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18292/consoleFull

@aarondav
Copy link
Contributor

Merging into master.

@asfgit asfgit closed this in ba28a8f Aug 11, 2014
asfgit pushed a commit that referenced this pull request Aug 15, 2014
@rxin
[SPARK-2936] Migrate Netty network module from Java to Scala
d3cce58 
The Netty network module was originally written when Scala 2.9.x had a bug that prevents a pure Scala implementation, and a subset of the files were done in Java. We have since upgraded to Scala 2.10, and can migrate all Java files now to Scala.

netty/netty#781

mesos/spark#522

Author: Reynold Xin <[email protected]>

Closes #1865 from rxin/netty and squashes the following commits:

332422f [Reynold Xin] Code review feedback
ca9eeee [Reynold Xin] Minor update.
7f1434b [Reynold Xin] [SPARK-2936] Migrate Netty network module from Java to Scala

(cherry picked from commit ba28a8f)
Signed-off-by: Reynold Xin <[email protected]>
xiliu82 pushed a commit to xiliu82/spark that referenced this pull request Sep 4, 2014
@rxin @conviva-zz
[SPARK-2936] Migrate Netty network module from Java to Scala
131749d 
The Netty network module was originally written when Scala 2.9.x had a bug that prevents a pure Scala implementation, and a subset of the files were done in Java. We have since upgraded to Scala 2.10, and can migrate all Java files now to Scala.

netty/netty#781

mesos/spark#522

Author: Reynold Xin <[email protected]>

Closes apache#1865 from rxin/netty and squashes the following commits:

332422f [Reynold Xin] Code review feedback
ca9eeee [Reynold Xin] Minor update.
7f1434b [Reynold Xin] [SPARK-2936] Migrate Netty network module from Java to Scala
szehon-ho pushed a commit to szehon-ho/spark that referenced this pull request Feb 7, 2024
@parthchandra
rdar://117273662 Update Boson to 0.3.20 (apache#1865)
9c8325e
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@rxin @SparkQA @aarondav @shivaram