[SPARK-5015] [mllib] Random seed for GMM + make test suite deterministic #3981
Conversation
|
Btw, in the docs, I did not say that the random seed is only used for initializing clusters since we may want to use it elsewhere in the future. |
|
LGTM. |
| assert(gmm.weight(0) ~== Ew absTol 1E-5) | ||
| assert(gmm.mu(0) ~== Emu absTol 1E-5) | ||
| assert(gmm.sigma(0) ~== Esigma absTol 1E-5) | ||
| } |
There was a problem hiding this comment.
For the single cluster case, I don't think the seed can make a difference. With only one cluster, this just makes sure the right mean and covariance are computed correctly.
There was a problem hiding this comment.
In general, you're correct. The issue is that, with some probability, the 5 data points chosen for initializing the cluster will be identical, causing initialization of the covariance matrix to fail.
I'd be fine with having this test use only 1 random seed instead of 5, but it's fast anyways.
|
Test build #25333 has finished for PR 3981 at commit
|
|
Merged into master. |
Issues:
Fix:
CC: @mengxr @tgaloppo