Lecture 13: About Simulation Optimization - Simulation and Optimization: Lecture Notes

Lectures 8 to 11 optimized models without randomness, and Lecture 12 simulated models with randomness for one fixed decision. Lecture 13 combines the two: we choose a decision whose performance is random and can only be evaluated by simulation. This notebook describes that setting, explains why it is harder than deterministic optimization, and gives an overview of four types of problems. The next notebooks discuss a strategy for each type: Comparing Scenarios, Ranking and Selection, Local Search and Gradient Methods.

Learning outcomes

On completion of this notebook, you will be able to:

describe the simulation optimization setting and give examples of it
explain why simulation optimization is harder than deterministic optimization
recognize which of the four types of simulation optimization problems a problem belongs to

The Setting¶

A real-life problem has random parameters $X$ (demands, travel times, arrival moments, ...) and a decision $\pi$ that we choose from a set of options $S$ . As in Why Simulate?, a model $r$ gives the random output $r(X, \pi)$ , and the goal is to maximize its expectation:

\max_{\pi \in S} \ E[r(X, \pi)].

(1)

The key feature is that $E[r(X, \pi)]$ can only be evaluated by simulating $r(X, \pi)$ : there is no closed-form expression for it that we could optimize directly. This is common in practice: for a queue, a production line or a project with random durations, we can simulate model realizations (Monte Carlo simulation or discrete-event simulation), but we cannot write down a formula for its expected performance. This setting is called simulation optimization, also known as optimization by simulation, simulation-based optimization, or simply simopt. Replacing $X$ by its expectation and solving $\max_{\pi \in S} r(EX, \pi)$ instead is not a way out: by the flaw of averages, $E[r(X, \pi)]$ and $r(EX, \pi)$ can be very different, and so can the decisions that maximize them.

Simulation optimization lets us optimize decisions for real-life processes, with all their randomness, also when no closed-form expression for the performance exists: should a bank open an extra desk, which layout of a factory gives the highest throughput, how long should a traffic light stay red?

Why Simulation Optimization Is Hard¶

In deterministic optimization, one evaluation of the objective gives the exact value of a solution. In simulation optimization, one simulation run gives only one random outcome $r(X, \pi)$ , and we need many runs to get a good estimate of $E[r(X, \pi)]$ for a single solution. This is time consuming, especially for a discrete-event simulation of a complex process, where a single run can already take a while. And however many runs we do, the estimates remain noisy, so we can never be sure that one solution is better than another: there are no optimality guarantees.

In practice, there is often a time limit within which a solution is needed, for example 5 minutes in an operational setting. With the time that one simulation run takes, this translates into a simulation budget $m$ : the total number of simulation runs we can do, say 10,000. The central question of this lecture is how to spend this budget well. Compare it with testing slot machines in a casino with 100 coins: how do you spend the coins to find the machine with the highest expected profit? Playing every machine equally often wastes coins on machines that are clearly bad, while playing only the machine that looked best after one try may miss a better one.

The noise in the estimates causes two problems. Because of the noise we might not recognize the signal and pick the wrong solution. And because of the noise we might overestimate the value: instead of selecting the solution with the highest value we pick the one with the highest random component, so the estimated value of the chosen solution is typically too high. To avoid this we could simulate each solution many more times, but this might take too much time. As a result, we cannot avoid these problems completely, but we can spend the budget better than by splitting it equally over all solutions. Ranking and Selection shows both problems for a newsvendor with 21 possible order sizes: with 200 runs per order size, the best order size is selected only about half of the time, and the estimated profit of the selected order size is too high on average.

Four Types of Problems¶

How we spend the simulation budget depends on the size and shape of $S$ . We write $|S|$ for the number of solutions in $S$ (the cardinality of $S$ ). Comparing $|S|$ with the budget $m$ gives four types of problems, each with its own strategy:

Shape of $S$	Strategy
$\lvert S \rvert = 2$	comparing scenarios
$S$ discrete and small: $m \gg \lvert S \rvert$ , so the budget suffices to simulate every solution many times	ranking and selection
$S$ discrete and large: $m \ll \lvert S \rvert$ , so the budget does not even suffice to simulate every solution once ( $S$ can also be infinite, e.g., $\{0, 1, 2, \dots\}$ )	local search
$S$ continuous (e.g., an interval)	gradient methods

For each type we give one simple method, to show the ideas. The scientific literature contains many more advanced methods: Fu (2002) is an accessible introduction to the subject, and Nelson (2013) is a textbook on simulation that includes a chapter on simopt. The techniques of these notebooks are also the building blocks to solve more challenging problems, for example problems with both discrete and continuous decisions, such as choosing the number of servers in a queue together with the length of their shifts.

References¶

Koole, G. (2019). An Introduction to Business Analytics. Chapter 8, “Simulation Optimization,” introduction.
Fu, M.C. (2002). “Optimization for simulation: Theory vs. practice.” INFORMS Journal on Computing, 14:192–215.
Nelson, B.L. (2013). Foundations and Methods of Stochastic Simulation. Springer.