A Max-Relevance-Min-Divergence criterion for data discretization with applications on naive Bayes

Shihe Wang, Jianfeng Ren, Ruibin Bai, Yuan Yao, Xudong Jiang

Research output: Journal PublicationArticlepeer-review

5 Citations (Scopus)

Abstract

In many classification models, data is discretized to better estimate its distribution. Existing discretization methods often target at maximizing the discriminant power of discretized data, while overlooking the fact that the primary target of data discretization in classification is to improve the generalization performance. As a result, the data tend to be over-split into many small bins since the data without discretization retain the maximal discriminant information. Thus, we propose a Max-Dependency-Min-Divergence (MDmD) criterion that maximizes both the discriminant information and generalization ability of the discretized data. More specifically, the Max-Dependency criterion maximizes the statistical dependency between the discretized data and the classification variable while the Min-Divergence criterion explicitly minimizes the JS-divergence between the training data and the validation data for a given discretization scheme. The proposed MDmD criterion is technically appealing, but it is difficult to reliably estimate the high-order joint distributions of attributes and the classification variable. We hence further propose a more practical solution, Max-Relevance-Min-Divergence (MRmD) discretization scheme, where each attribute is discretized separately, by simultaneously maximizing the discriminant information and the generalization ability of the discretized data. The proposed MRmD is compared with the state-of-the-art discretization algorithms under the naive Bayes classification framework on 45 benchmark datasets. It significantly outperforms all the compared methods on most of the datasets.

Original languageEnglish
Article number110236
JournalPattern Recognition
Volume149
DOIs
Publication statusPublished - May 2024

Keywords

  • Data discretization
  • Maximal dependency
  • Maximal relevance
  • Minimal divergence
  • Naive Bayes classification

ASJC Scopus subject areas

  • Software
  • Signal Processing
  • Computer Vision and Pattern Recognition
  • Artificial Intelligence

Fingerprint

Dive into the research topics of 'A Max-Relevance-Min-Divergence criterion for data discretization with applications on naive Bayes'. Together they form a unique fingerprint.

Cite this