Skip to content

[SPARK-8819] Fix build for maven 3.3.x #7219

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 3 commits into from
Closed

[SPARK-8819] Fix build for maven 3.3.x #7219

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
f39199c
Create a -Prelease profile that flags `createDependencyReducedPom`
Jul 5, 2015
3574ae4
Review comments
Jul 6, 2015
1d37e87
Merge branch 'master' of github.com:apache/spark into fix-maven-build
Jul 6, 2015
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions dev/create-release/create-release.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -118,13 +118,13 @@ if [[ ! "$@" =~ --skip-publish ]]; then

rm -rf $SPARK_REPO

build/mvn -DskipTests -Pyarn -Phive \
build/mvn -DskipTests -Pyarn -Phive -Prelease-profile\
-Phive-thriftserver -Phadoop-2.2 -Pspark-ganglia-lgpl -Pkinesis-asl \
clean install

./dev/change-version-to-2.11.sh

build/mvn -DskipTests -Pyarn -Phive \
build/mvn -DskipTests -Pyarn -Phive -Prelease-profile\
-Dscala-2.11 -Phadoop-2.2 -Pspark-ganglia-lgpl -Pkinesis-asl \
clean install

Expand Down
24 changes: 24 additions & 0 deletions pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -161,6 +161,8 @@
<fasterxml.jackson.version>2.4.4</fasterxml.jackson.version>
<snappy.version>1.1.1.7</snappy.version>
<netlib.java.version>1.1.2</netlib.java.version>
<!-- For maven shade plugin (see SPARK-8819) -->
<create.dependency.reduced.pom>false</create.dependency.reduced.pom>

<test.java.home>${java.home}</test.java.home>

Expand Down Expand Up @@ -1440,6 +1442,8 @@
<version>2.3</version>
<configuration>
<shadedArtifactAttached>false</shadedArtifactAttached>
<!-- Work around MSHADE-148. See SPARK-8819. -->
<createDependencyReducedPom>${create.dependency.reduced.pom}</createDependencyReducedPom>
<artifactSet>
<includes>
<!-- At a minimum we must include this to force effective pom generation -->
Expand Down Expand Up @@ -1826,6 +1830,26 @@
</properties>
</profile>

<profile>
<!--
Use this profile only for making Spark releases. Note that due to SPARK-8819,
you must use maven version 3.2.x or before to avoid running into MSHADE-148.
-->
<id>release-profile</id>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you rename this to just be called release? All of the profiles are, well, profiles. As a convention we don't include profile in the name.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually maybe release-publish is the best, since this is really specific to publishing the release to maven.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pwendell see my comments. release-profile is the standard name of the profile triggered by maven-release-plugin. I think you're not actually using the release plugin to publish (not an issue right now but did it not work? the script recreates some of what it does) but still might be clearer to use the standard name.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah - if this is a standard name we can use it. We don't actually use the release plugin though.

<properties>
<!--
The maven shade plugin has a bug where enabling the `createDependencyReducedPom`
property causes maven to go into an infinite loop (MSHADE-148). This is only an
issue for the Spark build if the maven version is 3.3.x or newer (SPARK-8819).

However, since disabling this property has the side effect of not resolving
variables in the released pom files (SPARK-8781), we need to enable this during
releases.
-->
<create.dependency.reduced.pom>true</create.dependency.reduced.pom>
</properties>
</profile>

<!--
These empty profiles are available in some sub-modules. Declare them here so that
maven does not complain when they're provided on the command line for a sub-module
Expand Down