Skip to content

[SPARK-4907][MLlib] Inconsistent loss and gradient in LeastSquaresGradient compared with R #3746

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 2 commits into from
Closed

[SPARK-4907][MLlib] Inconsistent loss and gradient in LeastSquaresGradient compared with R #3746

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
0b2c29c
first commit
Dec 19, 2014
19c2e85
make stepsize twice to converge to the same solution
Dec 19, 2014
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions mllib/src/main/scala/org/apache/spark/mllib/optimization/Gradient.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -94,16 +94,16 @@ class LogisticGradient extends Gradient {
* :: DeveloperApi ::
* Compute gradient and loss for a Least-squared loss function, as used in linear regression.
* This is correct for the averaged least squares loss function (mean squared error)
* L = 1/n ||A weights-y||^2
* L = 1/2n ||A weights-y||^2
* See also the documentation for the precise formulation.
*/
@DeveloperApi
class LeastSquaresGradient extends Gradient {
override def compute(data: Vector, label: Double, weights: Vector): (Vector, Double) = {
val diff = dot(data, weights) - label
val loss = diff * diff
val loss = diff * diff / 2.0
val gradient = data.copy
scal(2.0 * diff, gradient)
scal(diff, gradient)
(gradient, loss)
}

Expand All @@ -113,8 +113,8 @@ class LeastSquaresGradient extends Gradient {
weights: Vector,
cumGradient: Vector): Double = {
val diff = dot(data, weights) - label
axpy(2.0 * diff, data, cumGradient)
diff * diff
axpy(diff, data, cumGradient)
diff * diff / 2.0
}
}

Expand Down
6 changes: 3 additions & 3 deletions mllib/src/test/scala/org/apache/spark/mllib/regression/StreamingLinearRegressionSuite.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ class StreamingLinearRegressionSuite extends FunSuite with TestSuiteBase {
// create model
val model = new StreamingLinearRegressionWithSGD()
.setInitialWeights(Vectors.dense(0.0, 0.0))
.setStepSize(0.1)
.setStepSize(0.2)
.setNumIterations(25)

// generate sequence of simulated data
Expand Down Expand Up @@ -84,7 +84,7 @@ class StreamingLinearRegressionSuite extends FunSuite with TestSuiteBase {
// create model
val model = new StreamingLinearRegressionWithSGD()
.setInitialWeights(Vectors.dense(0.0))
.setStepSize(0.1)
.setStepSize(0.2)
.setNumIterations(25)

// generate sequence of simulated data
Expand Down Expand Up @@ -118,7 +118,7 @@ class StreamingLinearRegressionSuite extends FunSuite with TestSuiteBase {
// create model initialized with true weights
val model = new StreamingLinearRegressionWithSGD()
.setInitialWeights(Vectors.dense(10.0, 10.0))
.setStepSize(0.1)
.setStepSize(0.2)
.setNumIterations(25)

// generate sequence of simulated data for testing
Expand Down