Abstract

The solution to global (a posteriori) multi-objective optimization problems traditionally relies on population-based algorithms, which are very effective in generating a Pareto front. Unfortunately, due to the high number of function evaluations, these methods are of limited use in problems that involve expensive black-box functions. In recent years, multi-objective Bayesian optimization has emerged as a powerful alternative; however, in many applications, these methods fail to generate a diverse and well-spread Pareto front. To address this limitation, our work introduces a novel acquisition function (AF) for multi-objective Bayesian optimization that produces more informative acquisition landscapes. The proposed AF comprises two terms, namely, a distance-based metric and a diversity index. The distance-based metric, referred to as the expected Pareto distance change, promotes the evaluation of high-performing designs and repels low-performing design zones. The diversity term prevents the evaluation of designs that are similar to the ones contained in the current sampling plan. The proposed AF is studied using seven analytical problems and in the design optimization of sandwich composite armors for blast mitigation, which involves expensive finite element simulations. The results show that the proposed AF generates high-quality Pareto sets outperforming well-established methods such as the Euclidean-based expected improvement function. The proposed AF is also compared with respect to a recently proposed multi-objective approach. The difference in their performance is problem dependent.

References

1.
Shahriari
,
B.
,
Swersky
,
K.
,
Wang
,
Z.
,
Adams
,
R. P.
, and
De Freitas
,
N.
,
2015
, “
Taking the Human Out of the Loop: A Review of Bayesian Optimization
,”
Proc. IEEE
,
104
(
1
), pp.
148
175
.
2.
Frazier
,
P. I.
,
2018
, “
A Tutorial on Bayesian Optimization
,”
arXiv
. https://10.48550/arXiv.1807.02811
3.
Jones
,
D. R.
,
Schonlau
,
M.
, and
Welch
,
W. J.
,
1998
, “
Efficient Global Optimization of Expensive Black-Box Functions
,”
J. Global Optim.
,
13
(
4
), pp.
455
492
.
4.
Forrester
,
A.
,
Sobester
,
A.
, and
Keane
,
A.
,
2008
,
Engineering Design Via Surrogate Modelling: a Practical Guide
,
John Wiley & Sons
,
Chichester, UK
.
5.
Solak
,
E.
,
Murray-Smith
,
R.
,
Leithead
,
W.
,
Leith
,
D.
, and
Rasmussen
,
C.
,
2002
, “
Derivative Observations in Gaussian Process Models of Dynamic Systems
,”
Conference on Neural Information Processing Systems, NIPS 2002
,
Vancouver, British Columbia, Canada
,
Dec. 9–14
.
6.
Swersky
,
K.
,
Snoek
,
J.
, and
Adams
,
R. P.
,
2013
, “
Multi-task Bayesian Optimization
,”
Conference on Neural Information Processing Systems, NIPS 2013
,
Lake Tahoe, NV
,
Dec. 5–10
, pp.
2004
2012
.
7.
Álvarez
,
M. A.
, and
Lawrence
,
N. D.
,
2011
, “
Computationally Efficient Convolved Multiple Output Gaussian Processes
,”
J. Mach. Learn. Res.
,
12
(
41
), pp.
1459
1500
.
8.
Valladares
,
H.
, and
Tovar
,
A.
,
2020
, “
Design Optimization of Sandwich Composite Armors for Blast Mitigation Using Bayesian Optimization With Single and Multi-Fidelity Data
,”
SAE World Congress Experience, WCX 2020
,
Virtual-Only
,
Apr. 21–23
,
SAE Technical Paper 2020-01-0170
.
9.
Valladares
,
H.
,
Li
,
T.
,
Zhu
,
L.
,
El-Mounayri
,
H.
,
Hashem
,
A. M.
,
Abdel-Ghany
,
A. E.
, and
Tovar
,
A.
,
2022
, “
Gaussian Process-Based Prognostics of Lithium-Ion Batteries and Design Optimization of Cathode Active Materials
,”
J. Power. Sources.
,
528
, p.
231026
.
10.
Valladares
,
H.
, and
Tovar
,
A.
,
2022
, “
Nonlinear Multi-fidelity Bayesian Optimization: An Application in the Design of Blast Mitigating Structures
,”
SAE Int. J. Adv. Curr. Pract. Mobil.
,
4
(
6
), pp.
2248
2260
.
11.
Agrawal
,
S.
, and
Goyal
,
N.
,
2013
, “
Thompson Sampling for Contextual Bandits With Linear Payoffs
,”
International Conference on Machine Learning, ICML 2013
,
Atlanta, GA
,
June 16–21
, pp.
127
135
.
12.
Hennig
,
P.
, and
Schuler
,
C. J.
,
2012
, “
Entropy Search for Information-Efficient Global Optimization
,”
J. Mach. Learn. Res.
,
13
(
6
), pp.
1809
1837
.
13.
Frazier
,
P.
,
Powell
,
W.
, and
Dayanik
,
S.
,
2009
, “
The Knowledge-Gradient Policy for Correlated Normal Beliefs
,”
INFORMS J. Comput.
,
21
(
4
), pp.
599
613
.
14.
Knowles
,
J.
,
2006
, “
ParEGO: A Hybrid Algorithm With On-line Landscape Approximation for Expensive Multiobjective Optimization Problems
,”
IEEE Trans. Evol. Comput.
,
10
(
1
), pp.
50
66
.
15.
Zhang
,
Q.
,
Liu
,
W.
,
Tsang
,
E.
, and
Virginas
,
B.
,
2009
, “
Expensive Multiobjective Optimization by MOEA/D With Gaussian Process Model
,”
IEEE Trans. Evol. Comput.
,
14
(
3
), pp.
456
474
.
16.
Wagner
,
T.
,
Emmerich
,
M.
,
Deutz
,
A.
, and
Ponweiser
,
W.
,
2010
, “
On Expected-Improvement Criteria for Model-Based Multi-objective Optimization
,”
International Conference on Parallel Problem Solving from Nature, PPSN 2010
,
Krakow, Poland
,
Sept. 11–15
, Springer, pp.
718
727
.
17.
Emmerich
,
M.
,
Beume
,
N.
, and
Naujoks
,
B.
,
2005
, “
An EMO Algorithm Using the Hypervolume Measure as Selection Criterion
,”
International Conference on Evolutionary Multi-criterion Optimization, EMO 2005
,
Guanajuato, Mexico
,
Mar. 9–11
.
18.
Emmerich
,
M. T.
,
Giannakoglou
,
K. C.
, and
Naujoks
,
B.
,
2006
, “
Single- and Multiobjective Evolutionary Optimization Assisted by Gaussian Random Field Metamodels
,”
IEEE Trans. Evol. Comput.
,
10
(
4
), pp.
421
439
.
19.
Keane
,
A. J.
,
2006
, “
Statistical Improvement Criteria for Use in Multiobjective Design Optimization
,”
AIAA. J.
,
44
(
4
), pp.
879
891
.
20.
Svenson
,
J.
, and
Santner
,
T.
,
2016
, “
Multiobjective Optimization of Expensive-to-Evaluate Deterministic Computer Simulator Models
,”
Comput. Stat. Data Anal.
,
94
, pp.
250
264
.
21.
Wagner
,
T.
,
Beume
,
N.
, and
Naujoks
,
B.
,
2007
, “
Pareto-, Aggregation-, and Indicator-Based Methods in Many-Objective Optimization
,”
International Conference on Evolutionary Multi-criterion Optimization, EMO 2007
,
Matsushima and Sendai, Japan
,
Mar. 5–8
, Springer, pp.
742
756
.
22.
Couckuyt
,
I.
,
Deschrijver
,
D.
, and
Dhaene
,
T.
,
2014
, “
Fast Calculation of Multiobjective Probability of Improvement and Expected Improvement Criteria for Pareto Optimization
,”
J. Global Optim.
,
60
(
3
), pp.
575
594
.
23.
Li
,
M.
,
Li
,
G.
, and
Azarm
,
S.
,
2008
, “
A Kriging Metamodel Assisted Multi-objective Genetic Algorithm for Design Optimization
,”
ASME J. Mech. Des.
,
130
(
3
), p.
031401
.
24.
Shu
,
L.
,
Jiang
,
P.
,
Shao
,
X.
, and
Wang
,
Y.
,
2020
, “
A New Multi-objective Bayesian Optimization Formulation With the Acquisition Function for Convergence and Diversity
,”
ASME J. Mech. Des.
,
142
(
9
), p.
091703
.
25.
Wu
,
J.
, and
Azarm
,
S.
,
2001
, “
Metrics for Quality Assessment of a Multiobjective Design Optimization Solution Set
,”
ASME J. Mech. Des.
,
123
(
1
), pp.
18
25
.
26.
Paria
,
B.
,
Kandasamy
,
K.
, and
Póczos
,
B.
,
2020
, “
A Flexible Framework for Multi-objective Bayesian Optimization Using Random Scalarizations
,”
Proceedings of The 35th Uncertainty in Artificial Intelligence Conference, UAI 2019
, Vol. 115,
R.
Adams
and
V.
Gogate
, eds., Proceedings of Machine Learning Research (PMLR),
Tel Aviv, Israel
, pp.
766
776
.
27.
Tran
,
A.
,
Eldred
,
M.
,
McCann
,
S.
, and
Wang
,
Y.
,
2022
, “
srMO-BO-3GP: A Sequential Regularized Multi-objective Bayesian Optimization for Constrained Design Applications Using an Uncertain Pareto Classifier
,”
ASME J. Mech. Des.
,
144
(
3
), p.
031705
.
28.
Williams
,
C. K.
, and
Rasmussen
,
C. E.
,
2006
,
Gaussian Processes for Machine Learning
,
MIT Press Cambridge
,
MA
.
29.
Matthews
,
A. G. d. G.
,
Van Der Wilk
,
M.
,
Nickson
,
T.
,
Fujii
,
K.
,
Boukouvalas
,
A.
,
León-Villagrá
,
P.
,
Ghahramani
,
Z.
, and
Hensman
,
J.
,
2017
, “
GPflow: A Gaussian Process Library Using Tensor Flow
,”
J. Mach. Learn. Res.
,
18
(
40
), pp.
1
6
.
30.
Emmerich
,
M. T.
,
Deutz
,
A. H.
, and
Klinkenberg
,
J. W.
,
2011
, “
Hypervolume-Based Expected Improvement: Monotonicity Properties and Exact Computation
,”
IEEE Congress of Evolutionary Computation, CEC 2011
,
New Orleans, LA
,
June 5–8
,
IEEE
, pp.
2147
2154
..
31.
Abadi
,
M.
,
Agarwal
,
A.
,
Barham
,
P.
,
Brevdo
,
E.
,
Chen
,
Z.
,
Citro
,
C.
,
Corrado
,
G. S.
,
Davis
,
A.
,
Dean
,
J.
,
Devin
,
M.
,
Ghemawat
,
S.
,
Goodfellow
,
I.
,
Harp
,
A.
,
Irving
,
G.
,
Isard
,
M.
,
Jia
,
Y.
,
Jozefowicz
,
R.
,
Kaiser
,
L.
,
Kudlur
,
M.
,
Levenberg
,
J.
,
Mané
,
D.
,
Monga
,
R.
,
Moore
,
S.
,
Murray
,
D.
,
Olah
,
C.
,
Schuster
,
M.
,
Shlens
,
J.
,
Steiner
,
B.
,
Sutskever
,
I.
,
Talwar
,
K.
,
Tucker
,
P.
,
Vanhoucke
,
V.
,
Vasudevan
,
V.
,
Viégas
,
F.
,
Vinyals
,
O.
,
Warden
,
P.
,
Wattenberg
,
M.
,
Wicke
,
M.
,
Yu
,
Y.
, and
Zheng
,
X.
,
2015
, “
TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems
,” tensorflow.org.
32.
Zhao
,
G.
,
Dougherty
,
E.
,
Yoon
,
B.-J.
,
Alexander
,
F.
, and
Qian
,
X.
,
2021
, “
Efficient Active Learning for Gaussian Process Classification by Error Reduction
,”
Conference on Neural Information Processing Systems, NeurIPS 2021
,
Virtual-Only
,
Dec. 7–10
.
33.
Valladares
,
H.
, and
Tovar
,
A.
,
2022
, “
Multi-objective Bayesian Optimization Supported by Gaussian Process Classifiers and Conditional Probabilities
,”
International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, IDETC-CIE 2022
,
St. Louis, MO
,
Aug. 14–17
.
34.
Fonseca
,
C. M.
, and
Fleming
,
P. J.
,
1995
, “
An Overview of Evolutionary Algorithms in Multiobjective Optimization
,”
Evol. Comput.
,
3
(
1
), pp.
1
16
.
35.
Valladares
,
H.
, and
Tovar
,
A.
,
2020
, “
A Simple and Effective Methodology to Perform Multi-objective Bayesian Optimization: An Application in the Design of Sandwich Composite Armors for Blast Mitigation
,”
International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, IDETC-CIE 2020
,
Virtual-only conference
,
Aug. 17–19
, Vol. 84003, American Society of Mechanical Engineers, p. V11AT11A056.
36.
Kursawe
,
F.
,
1991
, “A Variant of Evolution Strategies for Vector Optimization,”
Parallel Problem Solving from Nature. PPSN 1990. Lecture Notes in Computer Science
, Vol.
496
,
H. P.
Schwefel
, and
R
Männer
, eds.,
Springer
,
Berlin/Heidelberg
, pp.
193
197
.
37.
Deb
,
K.
, and
Tiwari
,
S.
,
2008
, “
Omni-Optimizer: A Generic Evolutionary Algorithm for Single and Multi-objective Optimization
,”
Eur. J. Oper. Res.
,
185
(
3
), pp.
1062
1087
.
38.
Blank
,
J.
, and
Deb
,
K.
,
2020
, “
Pymoo: Multi-objective Optimization in Python
,”
IEEE Access
,
8
, pp.
89497
89509
.
39.
Zitzler
,
E.
,
Deb
,
K.
, and
Thiele
,
L.
,
2000
, “
Comparison of Multiobjective Evolutionary Algorithms: Empirical Results
,”
Evol. Comput.
,
8
(
2
), pp.
173
195
.
40.
Vlennet
,
R.
,
Fonteix
,
C.
, and
Marc
,
I.
,
1996
, “
Multicriteria Optimization Using a Genetic Algorithm for Determining a Pareto Set
,”
Int. J. Syst. Sci.
,
27
(
2
), pp.
255
260
.
41.
Deb
,
K.
,
Thiele
,
L.
,
Laumanns
,
M.
, and
Zitzler
,
E.
,
2005
, “Scalable Test Problems for Evolutionary Multiobjective Optimization,”
Evolutionary Multiobjective Optimization
,
A.
Abraham
,
L.
Jain
, and
R.
Goldberg
, eds.,
Springer
,
London
, pp.
105
145
.
42.
Huband
,
S.
,
Hingston
,
P.
,
Barone
,
L.
, and
While
,
L.
,
2006
, “
A Review of Multiobjective Test Problems and a Scalable Test Problem Toolkit
,”
IEEE Trans. Evol. Comput.
,
10
(
5
), pp.
477
506
.
43.
Duchi
,
J.
,
Hazan
,
E.
, and
Singer
,
Y.
,
2011
, “
Adaptive Subgradient Methods for Online Learning and Stochastic Optimization.
,”
J. Mach. Learn. Res.
,
12
(
7
), pp.
2121
2159
.
44.
Kingma
,
D. P.
, and
Ba
,
J.
,
2014
, “
Adam: A Method for Stochastic Optimization
,”,
arXiv:1412.6980v9
. https://arxiv.org/abs/1412.6980v9
45.
Crombecq
,
K.
,
De Tommasi
,
L.
,
Gorissen
,
D.
, and
Dhaene
,
T.
,
2009
, “
A Novel Sequential Design Strategy for Global Surrogate Modeling
,”
Winter Simulation Conference, WSC 2009
,
Austin, TX
,
Dec. 13–16
, IEEE, pp.
731
742
.
46.
Crombecq
,
K.
, and
Dhaene
,
T.
,
2010
, “
Generating Sequential Space-Filling Designs Using Genetic Algorithms and Monte Carlo Methods
,”
Conference on Simulated Evolution and Learning, SEAL 2010
,
Kanpur, India
,
Dec. 1–4
, Springer, pp.
80
84
.
47.
Williams
,
K.
,
McClennan
,
S.
,
Durocher
,
R.
,
St-Jean
,
B.
, and
Tremblay
,
J.
,
2002
, “
Validation of a Loading Model for Simulating Blast Mine Effects on Armoured Vehicles
,”
The 7th International LS-DYNA Users Conference
,
Dearborn, MI
,
May 19–21
.
48.
Vázquez
,
R.
,
2016
, “
A New Design for the Implementation of Isogeometric Analysis in Octave and Matlab: GeoPDEs 3.0
,”
Comput. Math. Appl.
,
72
(
3
), pp.
523
554
.
49.
Valladares
,
H.
, and
Tovar
,
A.
,
2021
, “
Multilevel Design of Sandwich Composite Armors for Blast Mitigation Using Bayesian Optimization and Non-uniform Rational B-splines
,”
SAE Int. J. Adv. Curr. Pract. Mobil.
,
3
(
4
), pp.
2146
2158
.
50.
Paszke
,
A.
,
Gross
,
S.
,
Massa
,
F.
,
Lerer
,
A.
,
Bradbury
,
J.
,
Chanan
,
G.
,
Killeen
,
T.
, et al
,
2019
, “
Pytorch: An Imperative Style, High-Performance Deep Learning Library
,”
Conference on Neural Information Processing Systems, NeurIPS 2019
,
Vancouver, British Columbia, Canada
,
Dec. 9–14
.
51.
Balandat
,
M.
,
Karrer
,
B.
,
Jiang
,
D.
,
Daulton
,
S.
,
Letham
,
B.
,
Wilson
,
A. G.
, and
Bakshy
,
E.
,
2020
, “
BoTorch: A Framework for Efficient Monte-Carlo Bayesian Optimization
,”
Conference on Neural Information Processing Systems, NeurIPS 2020
,
Virtual-only
,
Dec. 6–12
.
52.
Astudillo
,
R.
, and
Frazier
,
P.
,
2019
, “
Bayesian Optimization of Composite Functions
,”
International Conference on Machine Learning, ICML 2019
,
Long Beach, CA
,
June 9–15
, pp.
354
363
.
You do not currently have access to this content.