TD-MTP · Proposition 2

What a finite graph can reach

The structured planner builds a candidate by picking one control point per layer and interpolating between them. Whether a candidate can land in a given region of action space depends on which control points the graph holds, and drawing more paths does not change that.

01 · how a candidate is built

One node per layer, then interpolate

The graph holds M = 3 layers of N = 16 control points. A path picks an index independently in each layer and the interpolation matrix Φ turns those points into H = 3 actions. Every row of Φ is non-negative and sums to one, so each action is a weighted average of the points the path chose.

a₀ = U₁ a₁ = ½(U₁ + U₂) a₂ = U₂ Φ rows = (1, 0, 0), (½, ½, 0), (0, 1, 0)

At the reported setting the third layer gets zero weight in every row, so it is drawn but never used. As a result the 16³ = 4096 paths collapse to at most 16² = 256 distinct action sequences, and the middle action is always the midpoint of the two endpoints.

02 · the event

A sufficient condition for landing in a region

Take a box A in action space and consider sequences whose actions all lie inside it. If every control point a path selects is in A, every interpolated action is in A as well, because each action is a convex combination of those points and a box is convex.

Layers are chosen independently, so the probability of this sufficient event is the product of the per-layer fractions.

q_m = (nodes of layer m inside A) / N p_C = ∏_{m active} q_m Pr(at least one of n paths lands in the region) ≥ 1 − (1 − p_C)ⁿ

03 · interactive bound

The bound against simulated paths

Set how many of each layer’s sixteen points fall inside the region and how many paths the planner draws this iteration. The bound is a lower bound on the probability of at least one hit. The simulation draws that many paths from a graph with this arrangement and counts how many land inside.

points of layer 1 in A6 / 16 points of layer 2 in A5 / 16 paths drawn this iteration24

p_C = q₁q₂

—

bound on Pr(≥1 hit)

—

expected hits ≥

—

simulated hits

—

The simulated count comes from a single draw and varies around the expected value. The bound on the probability of at least one hit holds for any arrangement of points with these per-layer counts.

04 · the case H = M = 3

The bound is exact at the reported setting

With H = M = 3 the only actions produced are U₁, the midpoint, and U₂. If both endpoints are inside the box, the midpoint is too. If either endpoint is outside, it is itself one of the actions, so the sequence is already outside. The sufficient event is therefore also necessary, and the inequality holds with equality.

This does not hold in general. If a third layer has some weight, a path can enter the region even when the point it picks in one active layer lies outside, because a small weight on that point can be offset by the others. The product then underestimates the probability, and when any active q_m is zero the bound is zero even though some paths reach the region.

A numerical check over 600 random graphs, enumerating every path instead of sampling, found no violation of the bound, exact equality in every H = M = 3 case, and 76 of the 600 general cases where the bound is zero but some paths reach the region.

05 · computing q_m

q_m must be computed from whole points

The fractions count whole control points. Computing a fraction for each coordinate and multiplying them gives the wrong value, as the two-point example below shows.

coordinate 1 inside

1 / 2

coordinate 2 inside

1 / 2

product of marginals

0.25

actual q_m

0.00

Each coordinate has half its values inside the box, so the per-coordinate product predicts that a quarter of the points are inside, but neither point is. The permutations used to build the grid make coordinates independent at construction time, which says nothing about the points of the one fixed graph that paths are sampled from.

06 · scope

What the proposition does not cover

It concerns proposals before scoring. It says nothing about which candidate the elite fit selects, what the agent executes, or whether learning improves.
The region need not relate to the task. It is any box in action space, which may be infeasible under the dynamics or not worth reaching.
The count n is the number of tensor candidates drawn at that iteration, and the schedule takes it to zero at the midpoint of training. From then on the bound is 1 − (1 − p)⁰ = 0 because the mechanism it describes is switched off.
Coverage does not imply benefit. A structured proposal imposes a temporal constraint that can help search or exclude useful sequences, and the paper’s own controls measure the isolated contribution of these proposals at −14.6, with an interval that contains zero.

This page replaces an earlier explainer built on a proposition about reaching a second basin. That proposition is not in the current manuscript. The one shown here replaced it and makes a considerably weaker claim.