Exception in KPCA

[mhusinsky]

hello cesar,

when applying a transform to a double[] the Transform function throws an exception. try this adapted code from the documentation using jagged arrays:

double[][] sourceMatrix = new double[][]
        {
            new double[] { 2.5,  2.4 },
            new double[] { 0.5,  0.7 },
            new double[] { 2.2,  2.9 },
            new double[] { 1.9,  2.2 },
            new double[] { 3.1,  3.0 },
            new double[] { 2.3,  2.7 },
            new double[] { 2.0,  1.6 },
            new double[] { 1.0,  1.1 },
            new double[] { 1.5,  1.6 },
            new double[] { 1.1,  0.9 }
        };

        // Create a new linear kernel
        IKernel kernel = new Linear();

        // Creates the Kernel Principal Component Analysis of the given data
        var kpca = new KernelPrincipalComponentAnalysis(sourceMatrix, kernel);

        // Compute the Kernel Principal Component Analysis
        kpca.Compute();

        // The following statement throws an exception:
        // An unhandled exception of type 'System.IndexOutOfRangeException' occurred in Accord.Math.dll
        double[] transformed = kpca.Transform(sourceMatrix[0]);

when using a plain PrincipalComponentAnalysis everything runs fine as expected.
is this a bug?

thank you,
matthias

[cesarsouza]

Hi Matthias,

Yes, thanks for catching this bug! I will be committing a fix in a few minutes and perhaps generating a new NuGet package. In the meantime, do you think it would be possible for you to use the multidimensional version instead? If you need you can convert to and from jagged and multidimensional arrays using the extension methods ToMatrix and ToArray.

Hope it helps!
Cesar

[mhusinsky]

hi cesar,

thanks for your quick reply and fix! using the extension methods to convert to a matrix suffices as well.

there is one more thing that might be a bug (and is maybe related):
when applying a PCA to the data above, i will get 2 eigenvectors with 2 dimensions (as expected).
doing the same with a KPCA (and linear kernel) results in 10 eigenvectors with 2 dimensions each (which i don't expect as the results should be the same). maybe there is some kind of matrix dimension mismatch in the background?

matthias

[cesarsouza]

Hi Matthias!

Sorry, I missed your follow up to the question. In this case, the KPCA with a linear kernel will generate the same projection, but it doesn't use the same internal procedures. With KPCA, the PCA is done using the Kernel matrix, and the kernel matrix is always N x N where N is the number of samples. As such, the Eigenvalues/Eigenvector matrices are thus based on the Kernel matrix instead of the original samples, meaning we will be able to find up to N Eigenvectors and Eigenvalues during the transform. Now, for the linear kernel the extra Eigenvectors can be discarded, but this might not be true for other kernels.

In any case, you can also use the Threshold property of KPCA to discard unnecessary vectors after the transformation.

I will let the issue open until I check the original papers again (it has been some time!) if this is indeed correct. Thanks for reporting the issue!

Best regards,
Cesar

[cesarsouza]

Yes, this is the expected behavior. Again, the kpca.Threshold property can be used to cut some of the eigenvectors that don't contribute much to the variance.

[vincenzodentamaro]

Dear Cesàr,
I am trying to use the following code to filter out the noise and calculate the pre-image.

public static void Main (string[] args)
{
        double[,] dataset = {{-75.089, -75.003, -75.489, -67.669, -70.752, -80.788}, {-83.768, -74.948, -75.414, -67.628, -70.563, -80.531}, {-83.0, -74.901, -75.32, -67.605, -70.471, -80.278}, {-82.838, -74.78, -75.309, -67.693, -70.277, -79.944}, {-82.761, -74.745, -75.27, -67.772, -70.131, -79.667}, {-82.549, -74.928, -75.189, -67.85, -70.004, -79.337}, {-82.406, -75.107, -75.102, -71.859, -69.875, -79.06}, {-82.241, -74.967, -75.041, -73.0, -69.777, -78.81}, {-82.087, -71.605, -74.966, -72.947, -69.701, -78.547}, {-82.125, -80.0, -74.901, -72.79, -69.556, -78.307}, {-82.142, -79.98, -74.852, -72.626, -69.54, -78.062}, {-83.768, -79.921, -74.82, -72.494, -69.561, -77.843}, {-83.768, -79.813, -74.801, -72.425, -69.491, -77.597}, {-83.768, -71.605, -74.805, -72.329, -69.397, -77.371}, {-83.768, -84.0, -74.775, -72.171, -69.309, -77.106}, {-83.768, -83.784, -74.715, -72.052, -69.269, -76.915}, {-84.0, -71.605, -74.665, -72.012, -69.218, -76.707}, {-83.763, -71.527, -74.601, -71.896, -69.12, -76.514}, {-83.768, -71.605, -74.55, -71.859, -69.077, -76.301}};

        IKernel kernel = new Gaussian(0.5);
        var kpca = new KernelPrincipalComponentAnalysis(dataset, kernel);

        kpca.Compute();

        double[,] forward = kpca.Result;
        double[,] projection = kpca.Transform(dataset,kpca.GetNumberOfComponents(0.9f));

        double[,] reversion = kpca.Revert(projection);
                   //double[,] reversion = kpca.Revert(forward);
        Console.WriteLine ("Hello World!");
}

The problem is that the values I get in the reversion are in magnitude very different form the initial dataset and also sometimes they have an opposite sign. So they are completely wrong in my opinion.
Maybe is my mistake....

Any suggestion?

Thanks

[vincenzodentamaro]

It happens only when centerInFeatureSpace is true.
Thanks

[cesarsouza]

This seems to have been fixed somewhere in 2016. Sorry for the delay in updating/closing the issue.