Skip to content

Commit 3014c29

Browse files
j4munozzzcclp
authored andcommitted
[MLLIB] org.apache.spark.mllib.util.SVMDataGenerator generates ArrayIndexOutOfBoundsException. I have found the bug and tested the solution.
## What changes were proposed in this pull request? Just adjust the size of an array in line 58 so it does not cause an ArrayOutOfBoundsException in line 66. ## How was this patch tested? Manual tests. I have recompiled the entire project with the fix, it has been built successfully and I have run the code, also with good results. line 66: val yD = blas.ddot(trueWeights.length, x, 1, trueWeights, 1) + rnd.nextGaussian() * 0.1 crashes because trueWeights has length "nfeatures + 1" while "x" has length "features", and they should have the same length. To fix this just make trueWeights be the same length as x. I have recompiled the project with the change and it is working now: [spark-1.6.1]$ spark-submit --master local[*] --class org.apache.spark.mllib.util.SVMDataGenerator mllib/target/spark-mllib_2.11-1.6.1.jar local /home/user/test And it generates the data successfully now in the specified folder. Author: José Antonio <[email protected]> Closes apache#13895 from j4munoz/patch-2. (cherry picked from commit a3c7b41) Signed-off-by: Sean Owen <[email protected]> (cherry picked from commit 24d59fb)
1 parent 0bb89d6 commit 3014c29

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

mllib/src/main/scala/org/apache/spark/mllib/util/SVMDataGenerator.scala

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -55,7 +55,7 @@ object SVMDataGenerator {
5555
val sc = new SparkContext(sparkMaster, "SVMGenerator")
5656

5757
val globalRnd = new Random(94720)
58-
val trueWeights = Array.fill[Double](nfeatures + 1)(globalRnd.nextGaussian())
58+
val trueWeights = Array.fill[Double](nfeatures)(globalRnd.nextGaussian())
5959

6060
val data: RDD[LabeledPoint] = sc.parallelize(0 until nexamples, parts).map { idx =>
6161
val rnd = new Random(42 + idx)

0 commit comments

Comments
 (0)