Naive Bayesian Classifier

• Problem with k-NN
  • Sometimes not a clear cut that a data point falls into a category

Bayes Theorem

$$ P(A|B) = \frac{P(A \cap B)}{P(B)} = \frac{P(B|A)P(A)}{P(B)}

$$

Naive Bayes Algorithm

• Naive: assumes all features independent & equally important

$$ \begin{aligned} P(A|B \cap C) &= \frac{P(B \cap C|A)P(A)}{P(B \cap C)} \propto P(B|A)P(C|A)P(A) \ &= \frac{P(B|A)P(C|A)P(A)}{P(B)P(C)} \end{aligned}

$$

Features with 0 Probability

Add a small count to each feature.