Naive Bayesian Classifier

A naive Bayesian classifier calculates the probability of a data item belonging to each of two or more classes based on its input vector values.

It is used in classification tasks where the goal is to assign data items to predefined classes based on observed features. The algorithm works by applying Bayes’ theorem, assuming that the input features are conditionally independent given the class label. This assumption is why it is called “naive” because it simplifies the computation by ignoring any possible correlations between features.

For example, in a medical diagnosis scenario, the classifier can predict the likelihood of a disease based on symptoms. Despite the “naive” assumption of feature independence, the classifier often performs well in practice, especially for text classification and spam detection.

The main features of the naive Bayesian classifier are its simplicity, efficiency, and effectiveness in high-dimensional data. It is important because it provides a probabilistic approach to classification that is easy to implement and interpret.

Different types of naive Bayesian classifier are used with different types of input values.

For categorical input data:

• If the input vector values are boolean (e.g. a text contains a given word), the probabilities are combined using the Bernoulli naive Bayes algorithm.

For quantitative input data:

• If the input vector values are scalar (e.g a text contains a given word a stated number of times), the probabilities are combined using the multinomial naive Bayes algorithm. Zero probabilities (which result when a given input value never predicts a given class in the training data) are mathematically incompatible with multinomial naive Bayes. They have to be replaced with small positive values using a technique called additive smoothing.

• If the input vector values are continuous with Gaussian distribution, i.e. the input values that predict each class are normally distributed around specific points on a scale, the mean and standard deviation for each class can be plugged into an equation that then calculates the probability of a given input value belonging to each class.