A design optimization procedure was implemented to resize the holes of a combustor liner for practical applications. A combustor geometry evaluated without an enclosure was to be reformulated within an enclosure. The objective functions of the combustor with enclosure involved targeting the flow splits of the combustor without enclosure. Latin Hypercube Sampling (LHS) design of experiments (DOE) was utilized to obtain at least a pure quadratic response surface (RS). These were computed using Genetic Aggregate (GA). These RS were, in turn, evaluated by a multiple objective genetic algorithm (MOGA) optimizer. The focus of this study was a small-scale cavity-stabilized combustor. Steady, compressible three-dimensional simulations are performed using a multi-phase Realizable k-ε Reynolds-averaged Navier-Stokes (RANS) approach. Combustion-turbulence interaction is modeled with flamelet progress variable (FPV) and β-presumed probability density function (PDF). There are eleven input and output parameters corresponding to the combustor hole sizes and associated mass flow rates. The RS obtained with GA were principally of the Kriging kind (with constant and linear trends and damped sinusoid and Gaussian kernels). A combustor hole mass flow rate was mainly determined by its hole size but was also influenced by the other holes. The combustor flow split non-linearity shows that increasing a hole size increases its mass flow rate, but simultaneously decreases another hole flow rate. This was also verified by sensitivity analysis. Due to this non-linearity, matching flow splits between geometry without and with enclosure is challenging and may not be possible for some situations. Thus, it is concluded that optimization of the combustor geometry without the enclosure is not the best route. Rather, it would be better for the geometry to be optimized with the enclosure included in order to account for flow separation and non-linear influence of the combustor holes on the flow field.