What is Bayesian classification?
Bayesian classification is a probabilistic approach to predicting the class of a data instance based on Bayes’ theorem. Rather than committing to a single hard rule, it reasons about how likely each class is given the observed features, and assigns the instance to the most probable one. Formally, it computes the posterior probability P(C | X) = [P(X | C) · P(C)] / P(X), where P(C) is the prior (how common the class is before seeing any evidence), P(X | C) is the likelihood (how well the class explains the observed features), and P(X) is the evidence (a normalizing term common to all classes). The classifier then selects the class with the highest posterior — the maximum a posteriori (MAP) decision, which reduces to a maximum likelihood choice when all classes are equally likely.
The most widely used form is the Naïve Bayes classifier, which makes one simplifying assumption: that features are conditionally independent given the class. This lets the likelihood factorize into a simple product of per-feature probabilities, making the method fast and effective even with limited training data. Categorical features are handled through frequency counts, with Laplace smoothing applied to avoid zero probabilities, while continuous features are modelled with a distribution such as the Gaussian density, using each class’s mean and variance (Gaussian Naïve Bayes). Despite the “naïve” independence assumption rarely holding exactly, the method performs remarkably well in practice — powering applications like spam filtering, text and document classification, and medical diagnosis — and serves as the foundation for richer probabilistic models such as Bayesian belief networks.
Points to remember
