Uncertainty Handling and Quality Assessment in Data Mining (Advanced Information and Knowledge Processing) [Hardcover] Vazirgiannis, Michalis; Halkidi, Maria and Gunopulos, Dimitrious

The recent explosive growth of our ability to generate and store data has created a need for new, scalable and efficient, tools for data analysis. The main focus of the discipline of knowledge discovery in databases is to address this need. Knowledge discovery in databases is the fusion of many areas that are concerned with different aspects of data handling and data analysis, including databases, machine learning, statistics, and algorithms. Each of these areas addresses a different part of the problem, and places different emphasis on different requirements. For example, database techniques are designed to efficiently handle relatively simple queries on large amounts of data stored in external (disk) storage. Machine learning techniques typically consider smaller data sets, and the emphasis is on the accuracy ofa relatively complicated analysis task such as classification. The analysis of large data sets requires the design of new tools that not only combine and generalize techniques from different areas, but also require the design and development ofaltogether new scalable techniques.

Data Mining Process.- 2.1 Introduction to the Main Concepts of Data Mining.- 2.2 Knowledge and Data Mining.- 2.2.1 Knowledge Discovery in Database vs Data Mining.- 2.3 The Data Mining Process.- 2.3.1 Data Mining Requirements.- 2.4 Classification of Data Mining Methods.- 2.5 Overview of Data Mining Tasks.- 2.5.1 Clustering.- 2.5.1.1 Overview of Clustering Algorithms.- 2.5.1.2 Comparison of Clustering Algorithms.- 2.5.2 Classification.- 2.5.2.1 Bayesian Classification.- 2.5.2.2 Decision Trees.- 2.5.2.3 Neural Networks.- 2.5.2.4 Nearest Neighbor Classification.- 2.5.2.5 Support Vector Machines (SVMs).- 2.5.2.6 Fuzzy Classification approaches.- 2.5.3 Induction of classification rules.- 2.5.4 Association Rules.- 2.5.5 Sequential Patterns.- 2.5.6 Time Series Similarity.- 2.5.7 Visualization and Dimensionality Reduction.- 2.5.8 Regression.- 2.5.9 Summarization.- 2.6 Summary.- References.- Quality Assessment in Data Mining.- 3.1 Introduction.- 3.2 Data Pre-processing and Quality Assessment.- 3.3 Evaluation of Classification Methods.- 3.3.1 Classification Model Accuracy.- 3.3.1.1 Alternatives to the Accuracy Measure.- 3.3.2 Evaluating the Accuracy of Classification Algorithms.- 3.3.2.1 McNemar’s Test.- 3.3.2.2 A Test for the Difference of Two Proportions.- 3.3.2.3 The Resampled Paired t Test.- 3.3.2.4 The k-fold Cross-validated Paired t Test.- 3.3.3 Interestingness Measures of Classification Rules.- 3.3.3.1 Rule-Interest Function.- 3.3.3.2 Smyth and Goodman’s J-Measure.- 3.3.3.3 General Impressions.- 3.3.3.4 Gago and Bento’s Distance Metric.- 3.4 Association Rules.- 3.4.1 Association Rules Interestingness Measures.- 3.4.1.1 Coverage.- 3.4.1.2 Support.- 3.4.1.3 Confidence.- 3.4.1.4 Leverage.- 3.4.1.5 Lift.- 3.4.1.6 Rule Templates.- 3.4.1.7 Gray and Orlowska’s Interestingness.- 3.4.1.8 Dong and Li’s Interestingness.- 3.4.1.9 Peculiarity.- 3.4.1.10 Closed Association Rules Mining.- 3.5 Cluster Validity.- 3.5.1 Fundamental Concepts of Cluster Validity.- 3.5.2 External and Internal Validity Indices.- 3.5.2.1 Hypothesis Testing in Cluster Validity.- 3.5.2.2 External Criteria.- 3.5.2.3 Internal Criteria.- 3.5.3 Relative Criteria.- 3.5.3.1 Crisp Clustering.- 3.5.3.2 Fuzzy Clustering.- 3.5.4 Other Approaches for Cluster Validity.- 3.5.5 An Experimental Study on cluster validity.- 3.5.5.1 A Comparative Study.- 3.6 Summary.- References.- Uncertainty Handling in Data Mining.- 4.1 Introduction.- 4.2 Basic Concepts on Fuzzy Logic.- 4.2.1 Fuzzy Set Theory.- 4.2.2 Membership Functions.- 4.2.2.1 Hypertrapezoidal Fuzzy Membership Functions.- 4.2.2.2 Joint Degree of Membership.- 4.2.3 Fuzzy Sets and Information Measures.- 4.3 Basic Concepts on Probabilistic Theory.- 4.3.1 Uncertainty Quantified Probabi1istically.- 4.3.1.1 Bayesian Theorem.- 4.4 Probabilistic and Fuzzy Approaches.- 4.5 The EM Algorithm.- 4.5.1 General Description of EM Algorithm.- 4.6 Fuzzy Cluster Analysis.- 4.6.1 Fuzzy C-Means and its Variants.- 4.6.2 Fuzzy C-Means for Object-Data.- 4.6.3 Fuzzy C-Means (FCM) Alternatives.- 4.6.4 Applying Fuzzy C-Means Methodology to Relational Data.- 4.6.5 The Fuzzy C-Means Algorithm for Relational data.- 4.6.5.1 Comments on FCM for Relational Data.- 4.6.6 Noise Fuzzy Clustering Algorithm.- 4.6.7 Conditional Fuzzy C-Means Clustering.- 4.7 Fuzzy Classification Approaches.- 4.7.1 Fuzzy Decision Trees.- 4.7.1.1 Building a Fuzzy Decision Tree.- 4.7.1.2 Inference for Decision Assignment.- 4.7.2 Fuzzy Rules.- 4.8 Managing Uncertainty and Quality in the Classification Process.- 4.8.1 Framework Description.- 4.8.2 Mapping to the Fuzzy Domain.- 4.8.2.1 Classification Space (CS).- 4.8.2.2 Classification Value Space (CVS).- 4.8.3 Information Measures for Decision Support.- 4.8.3.1 Class Energy Metric.- 4.8.3.2 Attribute Energy Metric.- 4.8.4 Queries & Decision Support.- 4.8.5 Classification Scheme Quality Assessment.- 4.9 Fuzzy Association Rules.- 4.9.1 Defining Fuzzy Sets.- 4.9.2 Fuzzy Association Rule Definition.- 4.9.2.1 Fuzzy Support.- 4.9.2.2 Fuzzy Confidence.- 4.9.2.3 Fuzzy Correlation.- 4.9.3 Mining Fuzzy Association Rules Algorithms.- 4.10 Summary.- References.- UMiner: A Data Mining System Handling Uncertainty and Quality.- 5.1 Introduction.- 5.2 UMiner Development Approach.- 5.3 System Architecture.- 5.4 UMiner’s Data Mining Tasks.- 5.5 Demonstration.- 5.5.1 Clustering process.- 5.6 Summary.- References.- Case Studies.- 6.1 Extracting Association Rules for Medical Data Analysis.- 6.2 The Mining Process.- 6.2.1 Collection of Data.- 6.2.2 Data Cleaning and Pre-processing.- 6.2.3 Further Analysis of Extracted Association Rules.- 6.3 Cluster Analysis of Epidemiological Data.- References.