Uncertainty Handling and Quality Assessment in Data Mining

Michalis Vazirgiannis

ISBN 10: 1852336552 / ISBN 13: 9781852336554

Editore: Springer London Jul 2003, 2003

Neuware - Uncertainty Handling and Quality Assessment in Data Mining provides an introduction to the application of these concepts in Knowledge Discovery and Data Mining. It reviews the state-of-the-art in uncertainty handling and discusses a framework for unveiling and handling uncertainty. Coverage of quality assessment begins with an introduction to cluster analysis and a comparison of the methods and approaches that may be used. The techniques and algorithms involved in other essential data mining tasks, such as classification and extraction of association rules, are also discussed together with a review of the quality criteria and techniques for evaluating the data mining results. This book presents a general framework for assessing quality and handling uncertainty which is based on tested concepts and theories. This framework forms the basis of an implementation tool, 'Uminer' which is introduced to the reader for the first time. This tool supports the key data mining tasks while enhancing the traditional processes for handling uncertainty and assessing quality. Aimed at IT professionals involved with data mining and knowledge discovery, the work is supported with case studies from epidemiology and telecommunications that illustrate how the tool works in 'real world' data mining projects. The book would also be of interest to final year undergraduates or post-graduate students looking at: databases, algorithms, artificial intelligence and information systems particularly with regard to uncertainty and quality assessment. Codice articolo 9781852336554

Riassunto:

The recent explosive growth of our ability to generate and store data has created a need for new, scalable and efficient, tools for data analysis. The main focus of the discipline of knowledge discovery in databases is to address this need. Knowledge discovery in databases is the fusion of many areas that are concerned with different aspects of data handling and data analysis, including databases, machine learning, statistics, and algorithms. Each of these areas addresses a different part of the problem, and places different emphasis on different requirements. For example, database techniques are designed to efficiently handle relatively simple queries on large amounts of data stored in external (disk) storage. Machine learning techniques typically consider smaller data sets, and the emphasis is on the accuracy ofa relatively complicated analysis task such as classification. The analysis of large data sets requires the design of new tools that not only combine and generalize techniques from different areas, but also require the design and development ofaltogether new scalable techniques.

Contenuti: Data Mining Process.- 2.1 Introduction to the Main Concepts of Data Mining.- 2.2 Knowledge and Data Mining.- 2.2.1 Knowledge Discovery in Database vs Data Mining.- 2.3 The Data Mining Process.- 2.3.1 Data Mining Requirements.- 2.4 Classification of Data Mining Methods.- 2.5 Overview of Data Mining Tasks.- 2.5.1 Clustering.- 2.5.1.1 Overview of Clustering Algorithms.- 2.5.1.2 Comparison of Clustering Algorithms.- 2.5.2 Classification.- 2.5.2.1 Bayesian Classification.- 2.5.2.2 Decision Trees.- 2.5.2.3 Neural Networks.- 2.5.2.4 Nearest Neighbor Classification.- 2.5.2.5 Support Vector Machines (SVMs).- 2.5.2.6 Fuzzy Classification approaches.- 2.5.3 Induction of classification rules.- 2.5.4 Association Rules.- 2.5.5 Sequential Patterns.- 2.5.6 Time Series Similarity.- 2.5.7 Visualization and Dimensionality Reduction.- 2.5.8 Regression.- 2.5.9 Summarization.- 2.6 Summary.- References.- Quality Assessment in Data Mining.- 3.1 Introduction.- 3.2 Data Pre-processing and Quality Assessment.- 3.3 Evaluation of Classification Methods.- 3.3.1 Classification Model Accuracy.- 3.3.1.1 Alternatives to the Accuracy Measure.- 3.3.2 Evaluating the Accuracy of Classification Algorithms.- 3.3.2.1 McNemar’s Test.- 3.3.2.2 A Test for the Difference of Two Proportions.- 3.3.2.3 The Resampled Paired t Test.- 3.3.2.4 The k-fold Cross-validated Paired t Test.- 3.3.3 Interestingness Measures of Classification Rules.- 3.3.3.1 Rule-Interest Function.- 3.3.3.2 Smyth and Goodman’s J-Measure.- 3.3.3.3 General Impressions.- 3.3.3.4 Gago and Bento’s Distance Metric.- 3.4 Association Rules.- 3.4.1 Association Rules Interestingness Measures.- 3.4.1.1 Coverage.- 3.4.1.2 Support.- 3.4.1.3 Confidence.- 3.4.1.4 Leverage.- 3.4.1.5 Lift.- 3.4.1.6 Rule Templates.- 3.4.1.7 Gray and Orlowska’s Interestingness.- 3.4.1.8 Dong and Li’s Interestingness.- 3.4.1.9 Peculiarity.- 3.4.1.10 Closed Association Rules Mining.- 3.5 Cluster Validity.- 3.5.1 Fundamental Concepts of Cluster Validity.- 3.5.2 External and Internal Validity Indices.- 3.5.2.1 Hypothesis Testing in Cluster Validity.- 3.5.2.2 External Criteria.- 3.5.2.3 Internal Criteria.- 3.5.3 Relative Criteria.- 3.5.3.1 Crisp Clustering.- 3.5.3.2 Fuzzy Clustering.- 3.5.4 Other Approaches for Cluster Validity.- 3.5.5 An Experimental Study on cluster validity.- 3.5.5.1 A Comparative Study.- 3.6 Summary.- References.- Uncertainty Handling in Data Mining.- 4.1 Introduction.- 4.2 Basic Concepts on Fuzzy Logic.- 4.2.1 Fuzzy Set Theory.- 4.2.2 Membership Functions.- 4.2.2.1 Hypertrapezoidal Fuzzy Membership Functions.- 4.2.2.2 Joint Degree of Membership.- 4.2.3 Fuzzy Sets and Information Measures.- 4.3 Basic Concepts on Probabilistic Theory.- 4.3.1 Uncertainty Quantified Probabi1istically.- 4.3.1.1 Bayesian Theorem.- 4.4 Probabilistic and Fuzzy Approaches.- 4.5 The EM Algorithm.- 4.5.1 General Description of EM Algorithm.- 4.6 Fuzzy Cluster Analysis.- 4.6.1 Fuzzy C-Means and its Variants.- 4.6.2 Fuzzy C-Means for Object-Data.- 4.6.3 Fuzzy C-Means (FCM) Alternatives.- 4.6.4 Applying Fuzzy C-Means Methodology to Relational Data.- 4.6.5 The Fuzzy C-Means Algorithm for Relational data.- 4.6.5.1 Comments on FCM for Relational Data.- 4.6.6 Noise Fuzzy Clustering Algorithm.- 4.6.7 Conditional Fuzzy C-Means Clustering.- 4.7 Fuzzy Classification Approaches.- 4.7.1 Fuzzy Decision Trees.- 4.7.1.1 Building a Fuzzy Decision Tree.- 4.7.1.2 Inference for Decision Assignment.- 4.7.2 Fuzzy Rules.- 4.8 Managing Uncertainty and Quality in the Classification Process.- 4.8.1 Framework Description.- 4.8.2 Mapping to the Fuzzy Domain.- 4.8.2.1 Classification Space (CS).- 4.8.2.2 Classification Value Space (CVS).- 4.8.3 Information Measures for Decision Support.- 4.8.3.1 Class Energy Metric.- 4.8.3.2 Attribute Energy Metric.- 4.8.4 Queries & Decision Support.- 4.8.5 Classification Scheme Quality Assessment.- 4.9 Fuzzy Association Rules.- 4.9.1 Defining Fuzzy Sets.- 4.9.2 Fuzzy Association Rule Definition.- 4.9.2.1 Fuzzy Support.- 4.9.2.2 Fuzzy Confidence.- 4.9.2.3 Fuzzy Correlation.- 4.9.3 Mining Fuzzy Association Rules Algorithms.- 4.10 Summary.- References.- UMiner: A Data Mining System Handling Uncertainty and Quality.- 5.1 Introduction.- 5.2 UMiner Development Approach.- 5.3 System Architecture.- 5.4 UMiner’s Data Mining Tasks.- 5.5 Demonstration.- 5.5.1 Clustering process.- 5.6 Summary.- References.- Case Studies.- 6.1 Extracting Association Rules for Medical Data Analysis.- 6.2 The Mining Process.- 6.2.1 Collection of Data.- 6.2.2 Data Cleaning and Pre-processing.- 6.2.3 Further Analysis of Extracted Association Rules.- 6.3 Cluster Analysis of Epidemiological Data.- References.

Dati bibliografici

Titolo: Uncertainty Handling and Quality Assessment ...
Casa editrice: Springer London Jul 2003
Data di pubblicazione: 2003
Legatura: Buch
Condizione: Neu