This open access book provides a wealth of hands-on examples that illustrate how hyperparameter tuning can be applied in practice and gives deep insights into the working mechanisms of machine learning (ML) and deep learning (DL) methods. The aim of the book is to equip readers with the ability to achieve better results with significantly less time, costs, effort and resources using the methods described here. The case studies presented in this book can be run on a regular desktop or notebook computer. No high-performance computing facilities are required.
The idea for the book originated in a study conducted by Bartz & Bartz GmbH for the Federal Statistical Office of Germany (Destatis). Building on that study, the book is addressed to practitioners in industry as well as researchers, teachers and students in academia. The content focuses on the hyperparameter tuning of ML and DL algorithms, and is divided into two main parts: theory (Part I) and application (Part II). Essential topics covered include: a survey of important model parameters; four parameter tuning studies and one extensive global parameter tuning study; statistical analysis of the performance of ML and DL methods based on severity; and a new, consensus-ranking-based way to aggregate and analyze results from multiple algorithms. The book presents analyses of more than 30 hyperparameters from six relevant ML and DL methods, and provides source code so that users can reproduce the results. Accordingly, it serves as a handbook and textbook alike.
Eva Bartz is an expert in law and data protection. Within the wide area of data protection, she specializes particularly in the application of artificial intelligence and its benefits and dangers. Based on this vast experience, she founded Bartz & Bartz GmbH in 2014 together with Thomas Bartz-Beielstein and offers consulting for a variety of customers. She translates the academic expertise of Bartz & Bartz GmbH’s advisors - who are leading experts in their fields - into a benefit for her customers. One of these customers was the Federal Statistical Office of Germany (Destatis), and the study for them laid the groundwork for this book.
Prof. Dr. Thomas Bartz-Beielstein is an artificial intelligence expert with 30+ years of experience. He is a professor of applied mathematics at TH Köln in Germany and the director of the Institute for Data Science, Engineering, and Analytics (IDE+A). His research lies in artificial intelligence, machine learning, simulation, and optimization. He developed the Sequential Parameter Optimization (SPO). SPO integrates approaches from surrogate model-based optimization and evolutionary computing. He has worked on diverse topics from applied mathematics and statistics, design of experiments, simulation-based optimization and applications in domains as water industry, elevator control, or mechanical engineering.
Prof. Dr. Martin Zaefferer is a professor at Duale Hochschule Baden-Württemberg Ravensburg, teaching subjects related to data science in business informatics. Previously, he worked as a consultant at Bartz & Bartz GmbH and as a researcher at TH Köln, where he also studied electrical engineering and automation. He received a PhD from the Department of Computer Science at TU Dortmund University. Subsequently, he developed a keen interest in researching methods from the intersection of optimization and machine learning algorithms. He is passionate about the analysis of complex processes and finding novel solutions to challenging real-world problems.
Prof. Dr. Olaf Mersmann is a professor of data science at TH Köln-University of Applied Sciences in Germany and a member of the Institute for Data Science, Engineering, and Analytics (IDE+A). Having studied physics, statistics and data science, his research interests include landscape analysis for black box optimization problems and industrial machine learning applications. He is one of the developers of the exploratory landscape analysis approach to characterize continuous function landscapes.
