Uncertainty Handling and Quality Assessment in Data Mining

Covers the key topics of assessing the quality of the results and handling uncertainty in data mining.

Data Mining Process.- 2.1 Introduction to the Main Concepts of Data Mining.- 2.2 Knowledge and Data Mining.- 2.2.1 Knowledge Discovery in Database vs Data Mining.- 2.3 The Data Mining Process.- 2.3.1 Data Mining Requirements.- 2.4 Classification of Data Mining Methods.- 2.5 Overview of Data Mining Tasks.- 2.5.1 Clustering.- 2.5.1.1 Overview of Clustering Algorithms.- 2.5.1.2 Comparison of Clustering Algorithms.- 2.5.2 Classification.- 2.5.2.1 Bayesian Classification.- 2.5.2.2 Decision Trees.- 2.5.2.3 Neural Networks.- 2.5.2.4 Nearest Neighbor Classification.- 2.5.2.5 Support Vector Machines (SVMs).- 2.5.2.6 Fuzzy Classification approaches.- 2.5.3 Induction of classification rules.- 2.5.4 Association Rules.- 2.5.5 Sequential Patterns.- 2.5.6 Time Series Similarity.- 2.5.7 Visualization and Dimensionality Reduction.- 2.5.8 Regression.- 2.5.9 Summarization.- 2.6 Summary.- References.- Quality Assessment in Data Mining.- 3.1 Introduction.- 3.2 Data Pre-processing and Quality Assessment.- 3.3 Evaluation of Classification Methods.- 3.3.1 Classification Model Accuracy.- 3.3.1.1 Alternatives to the Accuracy Measure.- 3.3.2 Evaluating the Accuracy of Classification Algorithms.- 3.3.2.1 McNemar’s Test.- 3.3.2.2 A Test for the Difference of Two Proportions.- 3.3.2.3 The Resampled Paired t Test.- 3.3.2.4 The k-fold Cross-validated Paired t Test.- 3.3.3 Interestingness Measures of Classification Rules.- 3.3.3.1 Rule-Interest Function.- 3.3.3.2 Smyth and Goodman’s J-Measure.- 3.3.3.3 General Impressions.- 3.3.3.4 Gago and Bento’s Distance Metric.- 3.4 Association Rules.- 3.4.1 Association Rules Interestingness Measures.- 3.4.1.1 Coverage.- 3.4.1.2 Support.- 3.4.1.3 Confidence.- 3.4.1.4 Leverage.- 3.4.1.5 Lift.- 3.4.1.6 Rule Templates.- 3.4.1.7 Gray and Orlowska’s Interestingness.- 3.4.1.8 Dong and Li’s Interestingness.- 3.4.1.9 Peculiarity.- 3.4.1.10 Closed Association Rules Mining.- 3.5 Cluster Validity.- 3.5.1 Fundamental Concepts of Cluster Validity.- 3.5.2 External and Internal Validity Indices.- 3.5.2.1 Hypothesis Testing in Cluster Validity.- 3.5.2.2 External Criteria.- 3.5.2.3 Internal Criteria.- 3.5.3 Relative Criteria.- 3.5.3.1 Crisp Clustering.- 3.5.3.2 Fuzzy Clustering.- 3.5.4 Other Approaches for Cluster Validity.- 3.5.5 An Experimental Study on cluster validity.- 3.5.5.1 A Comparative Study.- 3.6 Summary.- References.- Uncertainty Handling in Data Mining.- 4.1 Introduction.- 4.2 Basic Concepts on Fuzzy Logic.- 4.2.1 Fuzzy Set Theory.- 4.2.2 Membership Functions.- 4.2.2.1 Hypertrapezoidal Fuzzy Membership Functions.- 4.2.2.2 Joint Degree of Membership.- 4.2.3 Fuzzy Sets and Information Measures.- 4.3 Basic Concepts on Probabilistic Theory.- 4.3.1 Uncertainty Quantified Probabi1istically.- 4.3.1.1 Bayesian Theorem.- 4.4 Probabilistic and Fuzzy Approaches.- 4.5 The EM Algorithm.- 4.5.1 General Description of EM Algorithm.- 4.6 Fuzzy Cluster Analysis.- 4.6.1 Fuzzy C-Means and its Variants.- 4.6.2 Fuzzy C-Means for Object-Data.- 4.6.3 Fuzzy C-Means (FCM) Alternatives.- 4.6.4 Applying Fuzzy C-Means Methodology to Relational Data.- 4.6.5 The Fuzzy C-Means Algorithm for Relational data.- 4.6.5.1 Comments on FCM for Relational Data.- 4.6.6 Noise Fuzzy Clustering Algorithm.- 4.6.7 Conditional Fuzzy C-Means Clustering.- 4.7 Fuzzy Classification Approaches.- 4.7.1 Fuzzy Decision Trees.- 4.7.1.1 Building a Fuzzy Decision Tree.- 4.7.1.2 Inference for Decision Assignment.- 4.7.2 Fuzzy Rules.- 4.8 Managing Uncertainty and Quality in the Classification Process.- 4.8.1 Framework Description.- 4.8.2 Mapping to the Fuzzy Domain.- 4.8.2.1 Classification Space (CS).- 4.8.2.2 Classification Value Space (CVS).- 4.8.3 Information Measures for Decision Support.- 4.8.3.1 Class Energy Metric.- 4.8.3.2 Attribute Energy Metric.- 4.8.4 Queries & Decision Support.- 4.8.5 Classification Scheme Quality Assessment.- 4.9 Fuzzy Association Rules.- 4.9.1 Defining Fuzzy Sets.- 4.9.2 Fuzzy Association Rule Definition.- 4.9.2.1 Fuzzy Support.- 4.9.2.2 Fuzzy Confidence.- 4.9.2.3 Fuzzy Correlation.- 4.9.3 Mining Fuzzy Association Rules Algorithms.- 4.10 Summary.- References.- UMiner: A Data Mining System Handling Uncertainty and Quality.- 5.1 Introduction.- 5.2 UMiner Development Approach.- 5.3 System Architecture.- 5.4 UMiner’s Data Mining Tasks.- 5.5 Demonstration.- 5.5.1 Clustering process.- 5.6 Summary.- References.- Case Studies.- 6.1 Extracting Association Rules for Medical Data Analysis.- 6.2 The Mining Process.- 6.2.1 Collection of Data.- 6.2.2 Data Cleaning and Pre-processing.- 6.2.3 Further Analysis of Extracted Association Rules.- 6.3 Cluster Analysis of Epidemiological Data.- References.