Skip to content

[SPARK-10441][SQL] Save data correctly to json. #8597

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 1 commit into from
Closed

[SPARK-10441][SQL] Save data correctly to json. #8597

wants to merge 1 commit into from

Conversation

yhuai
Copy link
Contributor

@yhuai yhuai commented Sep 4, 2015

@SparkQA
Copy link

SparkQA commented Sep 4, 2015

Test build #41992 has finished for PR 8597 at commit bd9898a.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Sep 4, 2015

Test build #41993 has finished for PR 8597 at commit 57b1e5d.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

liancheng
liancheng reviewed Sep 4, 2015
View reviewed changes
sql/hive/src/test/scala/org/apache/spark/sql/sources/SimpleTextRelation.scala
CatalystTypeConverters.convertToScala(catalystValue, dataType)
val scalaV = CatalystTypeConverters.convertToScala(catalystValue, dataType)

scalaV
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Remove scalaV?

@liancheng
Copy link
Contributor

Generally looks good except for a few minor issues.

@SparkQA
Copy link

SparkQA commented Sep 4, 2015

Test build #42018 has finished for PR 8597 at commit 5436236.

  • This patch fails Scala style tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Sep 4, 2015

Test build #1720 has finished for PR 8597 at commit 5436236.

  • This patch fails Scala style tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
    • class DCT(JavaTransformer, HasInputCol, HasOutputCol):
    • class SQLTransformer(JavaTransformer):
    • class StopWordsRemover(JavaTransformer, HasInputCol, HasOutputCol):
    • case class LimitNode(limit: Int, child: LocalNode) extends UnaryLocalNode
    • case class UnionNode(children: Seq[LocalNode]) extends LocalNode

@SparkQA
Copy link

SparkQA commented Sep 5, 2015

Test build #42026 has finished for PR 8597 at commit ddc0f0d.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Sep 5, 2015

Test build #1721 has finished for PR 8597 at commit 1605010.

  • This patch passes all tests.
  • This patch does not merge cleanly.
  • This patch adds no public classes.

@yhuai yhuai mentioned this pull request Sep 8, 2015
Closed
yhuai
yhuai reviewed Sep 8, 2015
View reviewed changes
sql/catalyst/src/test/scala/org/apache/spark/sql/RandomDataGenerator.scala
} else {
None
}
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@JoshRosen Can you take a look at my change to RandomDataGenerator?

@cloud-fan
Copy link
Contributor

LGTM

@SparkQA
Copy link

SparkQA commented Sep 8, 2015

Test build #1725 has finished for PR 8597 at commit ddc0f0d.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Sep 8, 2015

Test build #42118 has finished for PR 8597 at commit a4067b9.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Sep 8, 2015

Test build #42119 has finished for PR 8597 at commit 7df91d6.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

liancheng
liancheng reviewed Sep 8, 2015
View reviewed changes
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/json/JacksonParser.scala
case (VALUE_STRING, DoubleType) =>
// Special case handling for NaN and Infinity.
val value = parser.getText
if (value.equals("NaN") || value.equals("Infinity") || value.equals("-Infinity")) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above.

@liancheng
Copy link
Contributor

A few minor issues. Otherwise LGTM.

@SparkQA
Copy link

SparkQA commented Sep 8, 2015

Test build #42136 has finished for PR 8597 at commit bda8c77.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@yhuai yhuai mentioned this pull request Sep 8, 2015
Closed
@marmbrus
Copy link
Contributor

marmbrus commented Sep 8, 2015

Thanks, merging to master.

@asfgit asfgit closed this in 7a9dcbc Sep 8, 2015
asfgit pushed a commit that referenced this pull request Sep 8, 2015
@yhuai
[SPARK-10441] [SQL] [BRANCH-1.5] Save data correctly to json.
7fd4674 
https://issues.apache.org/jira/browse/SPARK-10441

This is the backport of #8597 for 1.5 branch.

Author: Yin Huai <[email protected]>

Closes #8655 from yhuai/timestampJson-1.5.
@vanzin vanzin mentioned this pull request Sep 8, 2015
Closed
ashangit pushed a commit to ashangit/spark that referenced this pull request Oct 19, 2016
@yhuai
[SPARK-10441] [SQL] [BRANCH-1.5] Save data correctly to json.
f84f467 
https://issues.apache.org/jira/browse/SPARK-10441

This is the backport of apache#8597 for 1.5 branch.

Author: Yin Huai <[email protected]>

Closes apache#8655 from yhuai/timestampJson-1.5.

(cherry picked from commit 7fd4674)
@srowen srowen mentioned this pull request Feb 6, 2017
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@yhuai @SparkQA @liancheng @cloud-fan @marmbrus