NaiveBayes issue when probability is 0

[spoved1]

When a probability of 0 is run through GeneralDiscreteDistribution!LogProbabilityMassFunction the value returned by Math.Log is -Inf. Once -Inf is returned the Decide value is always 0 with (0,0) probabilities (at least in cases where there are two outputs).

--==Example==--
Training data:
v1,v2,v3,v4,v5,v6,v7,v8,Result
TRUE,TRUE,FALSE,TRUE,TRUE,FALSE,FALSE,FALSE,FALSE
TRUE,TRUE,TRUE,TRUE,TRUE,FALSE,FALSE,FALSE,FALSE
TRUE,FALSE,TRUE,TRUE,TRUE,FALSE,FALSE,TRUE,FALSE
TRUE,TRUE,TRUE,TRUE,TRUE,FALSE,FALSE,TRUE,FALSE
FALSE,FALSE,TRUE,TRUE,TRUE,FALSE,FALSE,TRUE,FALSE
TRUE,TRUE,TRUE,TRUE,FALSE,FALSE,FALSE,FALSE,FALSE
FALSE,TRUE,TRUE,FALSE,TRUE,FALSE,FALSE,FALSE,FALSE
TRUE,TRUE,TRUE,FALSE,TRUE,FALSE,FALSE,FALSE,FALSE
FALSE,TRUE,TRUE,FALSE,TRUE,FALSE,FALSE,TRUE,FALSE
FALSE,TRUE,TRUE,TRUE,TRUE,FALSE,FALSE,TRUE,FALSE
FALSE,TRUE,TRUE,FALSE,FALSE,FALSE,FALSE,FALSE,FALSE
TRUE,FALSE,FALSE,TRUE,FALSE,FALSE,FALSE,TRUE,TRUE
TRUE,TRUE,FALSE,TRUE,FALSE,FALSE,FALSE,TRUE,TRUE
TRUE,TRUE,TRUE,TRUE,FALSE,FALSE,FALSE,TRUE,TRUE
FALSE,TRUE,TRUE,TRUE,FALSE,TRUE,TRUE,TRUE,TRUE
TRUE,TRUE,FALSE,FALSE,FALSE,TRUE,TRUE,TRUE,TRUE
FALSE,TRUE,FALSE,FALSE,FALSE,TRUE,TRUE,TRUE,TRUE
TRUE,TRUE,TRUE,TRUE,FALSE,TRUE,TRUE,TRUE,TRUE
FALSE,FALSE,FALSE,FALSE,FALSE,TRUE,TRUE,TRUE,TRUE
TRUE,TRUE,FALSE,TRUE,FALSE,TRUE,TRUE,TRUE,TRUE
FALSE,TRUE,FALSE,TRUE,FALSE,TRUE,TRUE,TRUE,TRUE
FALSE,TRUE,TRUE,FALSE,FALSE,TRUE,TRUE,TRUE,TRUE

Test:
FALSE,FALSE,FALSE,FALSE,TRUE,TRUE,TRUE,TRUE returns FALSE with NaiveBayes.Probabilities returning (0, 0).

A fix that seems to work (although I don't know if it is the mathematically correct thing to do) is check if the probability is zero and if it is, use a very low probability close to zero instead.

if (probabilities[value] == 0.0) {
return Math.Log(0.001);
}

In the above example, this fix causes TRUE to be returned with probabilities of (0.833, 0.167).

[cesarsouza]

Fixed on release 3.3.0. An example can be found here:

    // Declare some boolean data
    bool[,] source = 
    {
        // v1,v2,v3,v4,v5,v6,v7,v8,result
        { true,  true,  false, true,  true,  false, false, false, false },
        { true,  true,  true,  true,  true,  false, false, false, false },
        { true,  false, true,  true,  true,  false, false, true,  false },
        { true,  true,  true,  true,  true,  false, false, true,  false },
        { false, false, true,  true,  true,  false, false, true,  false },
        { true,  true,  true,  true,  false, false, false, false, false },
        { false, true,  true,  false, true,  false, false, false, false },
        { true,  true,  true,  false, true,  false, false, false, false },
        { false, true,  true,  false, true,  false, false, true,  false },
        { false, true,  true,  true,  true,  false, false, true,  false },
        { false, true,  true,  false, false, false, false, false, false },
        { true,  false, false, true,  false, false, false, true,  true  },
        { true,  true,  false, true,  false, false, false, true,  true  },
        { true,  true,  true,  true,  false, false, false, true,  true  },
        { false, true,  true,  true,  false, true,  true,  true,  true  },
        { true,  true,  false, false, false, true,  true,  true,  true  },
        { false, true,  false, false, false, true,  true,  true,  true  },
        { true,  true,  true,  true,  false, true,  true,  true,  true  },
        { false, false, false, false, false, true,  true,  true,  true  },
        { true,  true,  false, true,  false, true,  true,  true,  true  },
        { false, true,  false, true,  false, true,  true,  true,  true  },
        { false, true,  true,  false, false, true,  true,  true,  true  },
    };

    // Transform to integers, then to jagged (matrix with [][] instead of [,])
    int[][] data = source.ToInt32().ToJagged();

    // Classification setup
    var inputs = data.Get(null, 0, 8); // select all rows, with cols 0 to 8
    var outputs = data.GetColumn(8);   // select last column

    // Create the learning algorithm
    var learner = new NaiveBayesLearning();

    // Configure it to use the Laplace Smoothing rule in order to avoid zero probabilities
    learner.Options.InnerOption.UseLaplaceRule = true;

    // Learn the Naive Bayes model
    var nb = learner.Learn(inputs, outputs);

    // Evaluation of a single point
    int[] point = new[] { false, false, false, false, true, true, true, true }.ToInt32();

    int classIndex = nb.Decide(sp); // 1
    double[] probs = nb.Probabilities(sp); // 0.015, 0.985