Lecture 11: Exercises - Simulation and Optimization: Lecture Notes

The smaller exercises embedded in the other Lecture 11 notebooks check what you just read. This notebook collects the larger exercises for Lecture 11: independent problems worth more time.

import math

import matplotlib.pyplot as plt
from matplotlib.patches import Circle, FancyArrowPatch

# colors that read on the light and the dark site theme
NODE_FILL, NODE_EDGE = "#d6dce5", "#5b9bd5"
EDGE_COLOR, TEXT_COLOR = "#8c8c8c", "#111827"
NODE_RADIUS = 0.2


def draw_graph(
    positions,
    arcs,
    highlight=(),
    highlight_color="#d62728",
    directed=True,
    both_ways=(),
    label_at=None,
    node_notes=None,
    cut_x=None,
    figsize=(6, 3.2),
):
    """Draw a graph with a number (label) on every arc.

    positions: node -> (x, y)
    arcs: (i, j) -> label; both_ways: arcs drawn with an arrow at both ends
    highlight: arcs drawn thick in highlight_color
    label_at: (i, j) -> position of the label along the arc (default 0.5)
    node_notes: node -> (text, dx, dy) for a box next to the node
    cut_x: x-coordinate of a vertical line that marks a cut

    The background is transparent and every text sits on its own box, so
    the figure reads the same in the light and the dark site theme.
    """
    label_at = label_at or {}
    node_notes = node_notes or {}
    fig, ax = plt.subplots(figsize=figsize, dpi=150)
    fig.patch.set_alpha(0)
    ax.patch.set_alpha(0)
    for (i, j), label in arcs.items():
        (x1, y1), (x2, y2) = positions[i], positions[j]
        on = (i, j) in highlight or (not directed and (j, i) in highlight)
        color = highlight_color if on else EDGE_COLOR
        style = "-"
        if directed:
            style = "<|-|>" if (i, j) in both_ways else "-|>"
        # start and end the arc at the border of the node circles
        length = math.hypot(x2 - x1, y2 - y1)
        dx = (x2 - x1) / length * NODE_RADIUS
        dy = (y2 - y1) / length * NODE_RADIUS
        arrow = FancyArrowPatch(
            (x1 + dx, y1 + dy),
            (x2 - dx, y2 - dy),
            arrowstyle=style,
            mutation_scale=14,
            color=color,
            lw=3 if on else 1.5,
            shrinkA=0,
            shrinkB=0,
        )
        ax.add_patch(arrow)
        t = label_at.get((i, j), 0.5)
        ax.text(
            x1 + t * (x2 - x1),
            y1 + t * (y2 - y1),
            str(label),
            ha="center",
            va="center",
            fontsize=11,
            color=TEXT_COLOR,
            bbox={"boxstyle": "round,pad=0.2", "fc": "white", "ec": color},
        )
    for node, (x, y) in positions.items():
        circle = Circle(
            (x, y), NODE_RADIUS, fc=NODE_FILL, ec=NODE_EDGE, lw=1.5, zorder=3
        )
        ax.add_patch(circle)
        ax.text(x, y, node, ha="center", va="center", fontsize=12, zorder=4)
    for node, (text, dx, dy) in node_notes.items():
        x, y = positions[node]
        ax.text(
            x + dx,
            y + dy,
            text,
            ha="center",
            va="center",
            fontsize=11,
            color=highlight_color,
            fontweight="bold",
            bbox={
                "boxstyle": "round,pad=0.2",
                "fc": "white",
                "ec": highlight_color,
            },
        )
    xs = [x for x, _ in positions.values()]
    ys = [y for _, y in positions.values()]
    if cut_x is not None:
        ax.plot(
            [cut_x, cut_x],
            [min(ys) - 0.4, max(ys) + 0.4],
            color="#d62728",
            lw=3,
        )
    ax.set_xlim(min(xs) - 0.45, max(xs) + 0.45)
    ax.set_ylim(min(ys) - 0.45, max(ys) + 0.45)
    ax.set_aspect("equal")
    ax.axis("off")
    plt.show()

Homework Exercises¶

draw_graph(
    {"A": (0, 1), "B": (1.2, 2), "C": (1.2, 0), "D": (2.8, 2), "E": (2.8, 0)},
    {
        ("A", "B"): 3,
        ("A", "C"): 1,
        ("B", "D"): 2,
        ("B", "E"): 4,
        ("C", "B"): 1,
        ("C", "E"): 5,
        ("D", "E"): 1,
    },
)

a. Solve this using Dijkstra’s algorithm.

b. Give the linear optimization formulation of this problem.

c. Change the LO formulation from part b to find the longest path from A to E that does not visit node C.

Solution to Exercise 1 #

a. Running Dijkstra’s algorithm from A (each row is one step; $d_W(x)$ is the shortest distance found so far to $x$ using only intermediate nodes in $W$ , with the node chosen as current underlined):

step	A	B	C	D	E	current	$W$
start	0	$\infty$	$\infty$	$\infty$	$\infty$	A	$\{\}$
1		3 (A)	1 (A)	$\infty$	$\infty$	C	$\{A\}$
2		2 (C)		$\infty$	6 (C)	B	$\{A, C\}$
3				4 (B)	6 (C)	D	$\{A, C, B\}$
4					5 (D)	E	$\{A, C, B, D\}$

The shortest path is $A \to C \to B \to D \to E$ , distance 5.

b. Let $x_{ij} = 1$ if edge $(i, j)$ is traversed, 0 otherwise. The LO is

\begin{aligned} \min \quad & 3x_{AB} + x_{AC} + 2x_{BD} + 4x_{BE} + x_{CB} + 5x_{CE} + x_{DE} \\ \text{s.t.} \quad & x_{AB} + x_{AC} = 1 \ \text{(source)} \\ & x_{BE} + x_{CE} + x_{DE} = 1 \ \text{(destination)} \\ & x_{AB} + x_{CB} = x_{BD} + x_{BE} \ \text{(node B)} \\ & x_{AC} = x_{CB} + x_{CE} \ \text{(node C)} \\ & x_{BD} = x_{DE} \ \text{(node D)} \\ & x_{ij} \ge 0 \text{ for all } i, j. \end{aligned}

(1)

Because the shortest-path problem has this special structure, requiring $x_{ij}$ to be binary is not necessary; the destination constraint is automatically satisfied by the others.

c. Turn the objective into a maximization and add $x_{AC} = 0$ (forbidding node C).

draw_graph(
    {"A": (0, 1), "B": (1.6, 2), "C": (1.6, 1), "D": (1.6, 0), "E": (3.2, 1)},
    {
        ("A", "B"): 6,
        ("A", "D"): 2,
        ("B", "C"): 1,
        ("B", "E"): 4,
        ("C", "D"): 2,
        ("D", "E"): 4,
    },
)

draw_graph(
    {"A": (0, 1), "B": (1.2, 2), "C": (1.2, 0), "D": (2.6, 1), "E": (3.8, 1)},
    {
        ("A", "B"): 2,
        ("A", "C"): 4,
        ("B", "C"): 1,
        ("B", "D"): 6,
        ("B", "E"): 5,
        ("C", "D"): 3,
        ("C", "E"): 2,
        ("D", "E"): 0,
    },
    # move the labels of the arcs into D and E apart
    label_at={
        ("B", "D"): 0.4,
        ("C", "D"): 0.4,
        ("B", "E"): 0.7,
        ("C", "E"): 0.7,
    },
)

a. Find the cheapest path from A to E using Dijkstra’s algorithm. Show your steps.

b. Give an LO problem for finding the cheapest path from A to E.

c. Show that the cheapest path from part a is a feasible solution of the LO from part b.

d. Give an LO problem for finding the longest path that visits node D.

Solution to Exercise 3 #

a. Dijkstra’s algorithm from A (previous node on the shortest path in square brackets):

step	A	B	C	D	E	current	$W$
start	0	$\infty$	$\infty$	$\infty$	$\infty$	A	$\{\}$
1		2 [A]	4 [A]	$\infty$	$\infty$	B	$\{A\}$
2			3 [B]	8 [B]	7 [B]	C	$\{A, B\}$
3				6 [C]	5 [C]	E	$\{A, B, C\}$

Reading backward from E gives the cheapest path $A \to B \to C \to E$ , cost 5.

b. Let $x_{ij} = 1$ if edge $(i, j)$ is traversed, 0 otherwise. The LO is

\begin{aligned} \min \quad & 2x_{AB} + 4x_{AC} + x_{BC} + 6x_{BD} + 5x_{BE} + 3x_{CD} + 2x_{CE} \\ \text{s.t.} \quad & x_{AB} + x_{AC} = 1 \ \text{(source A)} \\ & x_{AB} = x_{BC} + x_{BD} + x_{BE} \ \text{(node B)} \\ & x_{AC} + x_{BC} = x_{CD} + x_{CE} \ \text{(node C)} \\ & x_{BD} + x_{CD} = x_{DE} \ \text{(node D)} \\ & x_{BE} + x_{CE} + x_{DE} = 1 \ \text{(destination E)} \\ & x_{ij} \ge 0 \text{ for all } i, j. \end{aligned}

(3)

As in Exercise 1, requiring $x_{ij}$ to be binary is not necessary here.

c. Path $A \to B \to C \to E$ corresponds to $x_{AB} = x_{BC} = x_{CE} = 1$ and 0 otherwise. Substituting into each constraint: $1 = 1$ (source), $1 = 1 + 0 + 0$ (node B), $0 + 1 = 0 + 1$ (node C), $0 + 0 = 0$ (node D), $0 + 1 + 0 = 1$ (destination) — all satisfied, and together with nonnegativity this shows the solution is feasible.

d. Replace the $\min$ in part b with $\max$ , and add $x_{DE} = 1$ (equivalently, $x_{BD} + x_{CD} = 1$ ) to force the path through D.

draw_graph(
    {
        "A": (0, 1),
        "B": (1.2, 2),
        "C": (1.2, 0),
        "D": (2.8, 2),
        "E": (2.8, 0),
        "F": (4, 1),
    },
    {
        ("A", "B"): 3,
        ("A", "C"): 2,
        ("B", "D"): 3,
        ("C", "B"): 1,
        ("C", "D"): 2,
        ("C", "E"): 1,
        ("D", "E"): 2,
        ("D", "F"): 4,
        ("E", "F"): 2,
    },
)

a. Use the Ford-Fulkerson algorithm to find the optimal solution.

b. Determine a minimal cut and give its value.

draw_graph(
    {
        "A": (0, 1),
        "B": (1.2, 2),
        "C": (1.2, 0),
        "D": (2.8, 2),
        "E": (2.8, 0),
        "F": (4, 1),
    },
    {
        ("A", "B"): 20,
        ("A", "C"): 10,
        ("B", "C"): 5,
        ("B", "D"): 20,
        ("C", "E"): 20,
        ("D", "F"): 10,
        ("E", "D"): 10,
        ("E", "F"): 25,
    },
)

draw_graph(
    {
        "A": (2.0, 2.35),
        "B": (3.52, 1.49),
        "C": (2.94, 0.09),
        "D": (1.06, 0.09),
        "E": (0.48, 1.49),
    },
    {
        ("A", "B"): 5,
        ("A", "C"): 8,
        ("A", "D"): 3,
        ("A", "E"): 2,
        ("B", "C"): 4,
        ("B", "D"): 6,
        ("B", "E"): 6,
        ("C", "D"): 7,
        ("C", "E"): 9,
        ("D", "E"): 5,
    },
    directed=False,
)

Exercise 6

The function build_tour below constructs a tour for the traveling salesman problem on the complete graph in Figure 6.

distance = {
    ("A", "B"): 5, ("A", "C"): 8, ("A", "D"): 3, ("A", "E"): 2,
    ("B", "C"): 4, ("B", "D"): 6, ("B", "E"): 6,
    ("C", "D"): 7, ("C", "E"): 9,
    ("D", "E"): 5,
}


def edge_length(i, j):
    return distance.get((i, j), distance.get((j, i)))


def build_tour(nodes, start):
    tour = [start]
    unvisited = [node for node in nodes if node != start]
    while unvisited:
        current = tour[-1]
        chosen = unvisited[0]
        for node in unvisited:
            if edge_length(current, node) < edge_length(current, chosen):
                chosen = node
        print(current, "->", chosen, "length", edge_length(current, chosen))
        tour.append(chosen)
        unvisited.remove(chosen)
    edges = zip(tour, tour[1:] + tour[:1])
    length = sum(edge_length(i, j) for i, j in edges)
    print("tour", " -> ".join(tour + tour[:1]), "length", length)
    return tour, length


build_tour(["A", "B", "C", "D", "E"], "A")

Figure 6:Complete graph of Exercise 6, with distances along the edges.

a. Without running the code, determine what it prints.

b. Describe in your own words what build_tour does. Does it always return a shortest tour? Motivate your answer.

c. How does the running time of build_tour grow with the number of nodes $n$ ? Is it a polynomial algorithm?

d. Propose two ways to improve the tours that build_tour finds, and say how each changes the running time. Apply one of them to Figure 6.

Solution to Exercise 6 #

a. The code prints

A -> E length 2
E -> D length 5
D -> B length 6
B -> C length 4
tour A -> E -> D -> B -> C -> A length 25

The last edge, C-A of length 8, is not printed in the loop but is part of the length $2 + 5 + 6 + 4 + 8 = 25$ .

b. Starting at start, the function repeatedly travels to the closest node that has not been visited yet, until all nodes are visited, and then returns to the start. This is the nearest neighbor heuristic. It does not always return a shortest tour: B → C → D → A → E → B has length $4 + 7 + 3 + 2 + 6 = 22 < 25$ . Every step takes the shortest edge available at that moment, without looking at what that choice means for later steps. In the example, the cheap edges A-E and E-D leave the expensive edge C-A of length 8 for the end. So it is a heuristic, not an exact algorithm.

c. The while loop runs $n - 1$ times, once for every node added to the tour. In each repetition the for loop looks at all unvisited nodes, at most $n - 1$ , and removing the chosen node also takes at most $n$ steps. In total that is in the order of $n \cdot n = n^2$ operations, $O(n^2)$ , so the algorithm is polynomial. Compare this with the $n!$ growth of trying all tours.

d. Some possible improvements:

Run build_tour from every start node, or from several random start nodes, and keep the shortest tour. From every start node this takes $n$ times as long, $O(n^3)$ . For Figure 6, the start nodes A to E give lengths 25, 22, 23, 22 and 24, so starting in B (or D) gives the tour B → C → D → A → E → B of length 22, which is optimal here.
Look two nodes ahead: choose the next node $j$ such that $d_{\text{current},j} + d_{jk}$ is smallest over all pairs of unvisited nodes $j \ne k$ . Each step then looks at in the order of $n^2$ pairs, so the running time becomes $O(n^3)$ . From A, the pair E, D (length $2 + 5 = 7$ ) gives E first; from E, the pair B, C ( $6 + 4 = 10$ ) gives B; then C and D follow. The tour A → E → B → C → D → A has length $2 + 6 + 4 + 7 + 3 = 22$ .
Use the tour as initial tour for the 2-opt heuristic.

References¶

Koole, G. (2019). An Introduction to Business Analytics. §7.1, §7.3, §7.4 (shortest path, maximum flow).