Bayesian Optimization and Data Science (eBook)

Francesco Archetti, Antonio Candelieri (Autoren)

eBook Download: PDF

2019 | 1st ed. 2019
XIII, 126 Seiten
Springer International Publishing (Verlag)
978-3-030-24494-1 (ISBN)

Infos

This volume brings together the main results in the field of Bayesian Optimization (BO), focusing on the last ten years and showing how, on the basic framework, new methods have been specialized to solve emerging problems from machine learning, artificial intelligence, and system optimization. It also analyzes the software resources available for BO and a few selected application areas. Some areas for which new results are shown include constrained optimization, safe optimization, and applied mathematics, specifically BO's use in solving difficult nonlinear mixed integer problems. 

The book will help bring readers to a full understanding of the basic Bayesian Optimization framework and gain an appreciation of its potential for emerging application areas. It will be of particular interest to the data science, computer science, optimization, and engineering communities.

Inhalt

Preface 5
Who should read this book? 8
Contents 11
1 Automated Machine Learning and Bayesian Optimization 14
1.1 Automated Machine Learning 14
1.1.1 Motivation 14
1.1.2 Model Selection 16
1.1.3 Hyperparameter Optimization 17
1.1.4 Combined Algorithm Selection and Hyperparameter Optimization 19
1.1.5 Why Hyperparameter Optimization Is Important? 19
1.2 The Basic Structure of Bayesian Optimization 21
1.2.1 Sequential Model-Based Optimization 21
1.2.2 Surrogate Model 23
1.2.3 Acquisition Function 25
1.3 Automated Machine Learning for Predictive Analytics 27
References 30
2 From Global Optimization to Optimal Learning 32
2.1 A Priori Analysis of Global Optimization Strategies 33
2.2 Lipschitz Global Optimization (LGO) 34
2.3 Random Search 37
2.3.1 General Properties of Uniform Sampling 39
2.3.2 Cluster Analysis 39
2.3.3 Stopping Rules 39
2.4 Bandits, Active Learning and Bayesian Optimization 41
References 45
3 The Surrogate Model 49
3.1 Gaussian Processes 49
3.1.1 Gaussian Processes Regression 49
3.1.2 Kernel: The Data Geometry of Bayesian Optimization 52
3.1.3 Embedding Derivative Observations in the Gaussian Process 55
3.1.4 Numerical Instability 58
3.2 Thompson Sampling 59
3.3 Alternative Models 62
3.3.1 Random Forest 63
3.3.2 Neural Networks: Feedforward, Deep and Bayesian 65
References 67
4 The Acquisition Function 69
4.1 Traditional Acquisition Functions 69
4.1.1 Probability of Improvement 69
4.1.2 Expected Improvement 70
4.1.3 Upper/Lower Confidence Bound 72
4.2 New Acquisition Functions 74
4.2.1 Scaled Expected Improvement 74
4.2.2 Portfolio Allocation 74
4.2.3 Thompson Sampling 75
4.2.4 Entropy-Based Acquisition Functions 77
4.2.5 Knowledge Gradient 78
4.2.6 Look-Ahead 80
4.2.7 K-Optimality 81
4.3 Optimizing the Acquisition Function 82
References 83
5 Exotic Bayesian Optimization 85
5.1 Constrained Global Optimization 85
5.2 Support Vector Machine—Constrained Bayesian Optimization 88
5.3 Safe Bayesian Optimization 98
5.4 Parallel Bayesian Optimization 102
5.5 Multi-objective Bayesian Optimization 103
5.6 Multi-source and Multi-fidelity Bayesian Optimization 105
References 106
6 Software Resources 109
6.1 Open Source Software 109
6.2 Bayesian Optimization as a Service 113
6.3 Bayesian Optimization-Based Services for Hyperparameters Optimization 114
6.4 Test Functions and Generators 115
6.4.1 Survey and Site/Repository of Test Functions 115
6.4.2 Test Functions Generators 119
6.5 Non-Bayesian Global Optimization Software 119
References 120
7 Selected Applications 122
7.1 Overview of Applications 122
7.2 Smart Water 127
7.2.1 Leakage Localization 127
7.2.2 Pump Scheduling Optimization 129
References 133

Produkt-Details

• Erscheint lt. Verlag: 25.9.2019
• Reihe/Serie: SpringerBriefs in Optimization
• Reihe/Serie: SpringerBriefs in Optimization
• Zusatzinfo: XIII, 126 p. 52 illus., 39 illus. in color.
• Sprache: englisch
• Themenwelt: Informatik ► Theorie / Studium ► Künstliche Intelligenz / Robotik
• Themenwelt: Mathematik / Informatik ► Mathematik ► Statistik
• Schlagworte: acquisition functions • automatic algorithm configuration • Gaussian process • knowledge gradient • Marketing • MarkTech
• ISBN-10: 3-030-24494-6 / 3030244946
• ISBN-13: 978-3-030-24494-1 / 9783030244941