SPARK-4567. Make SparkJobInfo and SparkStageInfo serializable

[sryza]

No description provided.

[SparkQA]

Test build #23779 has started for PR 3426 at commit cb4b8d2.

• This patch merges cleanly.

[SparkQA]

Test build #23779 has finished for PR 3426 at commit cb4b8d2.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23779/
Test PASSed.

[pwendell]

LGTM

[JoshRosen]

LGTM, too. I'm going to merge this into master and branch-1.2, since I can't imagine how this could cause 1.2.0 RC regressions and it might as well make it into 1.2.0 so that Hive on Spark can use this.

[tigerquoll]

Heh @JoshRosen @sryza , should this patch include a serialVersionUID attribute on the classes to be serialized to make sure compiler quirks don't cause different UIDs to be generated for the classes?

[JoshRosen]

Aren't default serialVersionUIDs generated in a consistent way across all JVMs because the algorithm for generating them is part of the Java spec?

[JoshRosen]

Err, across all compilers?

[tigerquoll]

http://stackoverflow.com/questions/285793/what-is-a-serialversionuid-and-why-should-i-use-it seems to be a good summary of the pros and cons of this approach

[srowen]

The mapping from class structure to serialVersionUID is well defined. The mapping from source to class structure may not be 100% since the compiler can stick in synthetics and such.

That said adding serialVersionUID is the less conservative default. The failure mode of forgetting to detect and update serialized form changes is potentially silent corruption of data. The failure mode of not adding it is getting exceptions about incompatible changes when the two versions could be made compatible.

I advise doing this only if really necessary because it doesn't work otherwise. It is not something to add just because .