Comparing freight policy alternatives without a month-long study

Pull-quote: “A single run answers a question nobody asked. The question was always which alternative, against which other alternative.”

A freight policy question is a set of alternatives, so freight policy scenario comparison is the actual work. Nobody asks whether a truck-to-rail diversion happens in isolation. They ask which option moves the most freight off a congested corridor, what each costs, and which one still holds if diesel prices move against the plan. Running any one of those models has been possible for decades. Keeping a dozen variants comparable to each other is where the work goes.

This post is for a transportation planner, an agency analyst, or an engineer holding a freight question that will be argued over in public. The argument will not be about your solver. It will be about whether variant B ran on the same geography as variant A, whether the commodity definitions matched, and where the inputs came from.

By the end you will know what a scenario has to be as a stored record, why branching and lineage are prerequisites rather than conveniences, which of seven simulation families answers which question, and why the data underneath decides whether your result survives review.

What you will be able to do

• Describe a scenario as a record rather than a model run, and name what that record holds.
• Explain why branching and lineage are what make a set of variants comparable at all.
• Choose between a side-by-side comparison and a parameter sweep for a given question.
• Match each of seven simulation families to the question it answers.
• Tell a single-lane elasticity heuristic apart from a multi-mode simulation.
• Say why public federal data changes what happens when a result is challenged.

What is a freight scenario, if it is not just a model run?

A scenario is a record: the assumptions you set, the shocks you apply, and the horizon year you project to, stored so that it can be re-read, copied, and compared months later. The run is what the record produces. It is not what the scenario is.

The distinction sounds academic until you have twelve variants. When a scenario lives in a notebook, the parameters live in the analyst’s head and in the notebook’s cell order. Six weeks later, nobody can say whether the rail diversion case used the 2035 or the 2040 horizon, or whether the fuel assumption moved for that variant only. The analysis was not wrong, it became unciteable, which in a public process amounts to the same thing.

Comparability also depends on both sides sharing a unit of analysis. In FreightCortex, those units are explicit: lanes as origin-to-destination zone pairs, corridors identified by code, Freight Analysis Framework zones for geography, and commodities expressed through SCTG codes with a NAICS concordance. Two runs that disagree on any one of those answer two different questions and then get printed side by side as though they answered one. Scenarios there are first-class records with the lifecycle you would expect of a durable object: create, read, update, delete, branch, and a lineage trail.

You know the record is doing its job when you can hand a colleague two scenario identifiers and they can state what differs between them without asking you.

Why is freight policy scenario comparison the hard part?

Because comparability is a property of the collection rather than of any individual run, and nothing about running a model well produces it. Each run can be defensible while the set is worthless.

Consider what diverges across an ordinary working week. One variant picks up a corrected base-year table after someone finds an error. Another gets a different commodity aggregation because the analyst wanted a cleaner chart. A third ran before the network file was updated. Each decision was reasonable in the moment, and together they mean the differences you are about to attribute to policy are partly bookkeeping, with no way to tell which part is which.

The traditional answer is procedural: a scoped study, a controlled data build, a documented methodology, a report at the end. What makes that route expensive is rarely the modelling. It is the assembly, the version control done by hand, and the week-five question that means rebuilding part of the work. Making the scenario the durable object changes what that question costs, because it becomes a branch instead of a rebuild.

Why do branching and lineage matter once you have many variants?

Branching lets you copy a scenario and change exactly one thing, and lineage records what the copy came from. Together they let you attribute a difference in output to a difference in input. Without both, you have a folder of results and an argument.

Branching is the discipline of one change per variant. A fuel shock variant must differ from its parent in the fuel assumption and nothing else. That is elementary experimental practice, and it is what a spreadsheet workflow quietly violates, because copying a file leaves no record of what you altered afterwards.

Lineage is the parent link. It answers the question that arrives late and matters most: where did this number come from? A scenario that knows its parent, and a parent that knows its own parent, gives you a chain back to a baseline whose assumptions were agreed with the people who live with the result. When a reviewer challenges the seventh variant, you show the chain.

Which comparison primitive do you need, side by side or a sweep?

Use a side-by-side comparison when the alternatives are discrete policy choices, and a parameter sweep when one input is continuous and you want to know where the answer changes. Reaching for the wrong one is how analyses end up recommending a single number nobody can defend.

Side by side is the primitive for “which of these”: three funding options, two alignment choices, a build case against a no-build case. FreightCortex exposes it as compare_scenarios, which puts completed runs next to each other instead of making a person export each one and reconcile the columns by hand. What comes out is a table in which the tradeoff is visible, which beats a declared winner and is harder to argue with.

A sweep is the primitive for “how sensitive”. Sensitivity analysis holds the scenario fixed and moves one parameter across a range, so you learn whether the conclusion flips at a 10 percent fuel increase or survives to 40. That is often the more useful finding, because it tells a decision maker how much of their judgment rides on an assumption they do not control.

Which simulation family answers which question?

Seven simulation families are shipped in FreightCortex and callable through both the API and an MCP server: cross elasticity, gravity model, growth forecast, network disruption, Monte Carlo, nested logit, and economic shock. Each answers a different question, and choosing the wrong one produces a technically valid result that does not address what was asked.

Family	The question it answers
Cross elasticity	How does demand for one mode respond when the price of another mode changes?
Gravity model	How do flows distribute across origin and destination pairs given size and distance?
Growth forecast	What volume sits on this lane in the horizon year if current trends continue?
Network disruption	What moves, and where, when a link or node degrades or closes?
Monte Carlo	What is the spread of outcomes when the inputs are genuinely uncertain?
Nested logit	Which mode is chosen when the alternatives are correlated and grouped?
Economic shock	What happens under movements in GDP, fuel prices, or trade policy?

Two deserve a note on how to read them. A Monte Carlo run produces a distribution, and reporting only its mean throws away the reason you ran it; the useful output is the shape of the tail, because a policy that is good on average and catastrophic in the bottom decile is a bad policy. A nested logit result is conditional on the nesting structure you declared, so the nesting belongs in the write-up rather than in an appendix.

Model Context Protocol, the open standard that lets an AI agent call a tool with structured arguments, means an agent can request a run the same way a person does. That changes nothing about the discipline above. An agent branching scenarios without recording what changed produces the same unciteable pile, faster.

When is a single-lane what-if enough, and when do you need a full simulation?

A single-lane what-if and a multi-mode simulation are different instruments for different questions, and FreightCortex keeps them at separate endpoints for exactly that reason. POST /v1/ops/whatif is documented as a single-lane elasticity heuristic. Multi-mode work goes through POST /v1/scenarios.

The heuristic is the right tool for a question asked in a meeting. One lane, one lever, a directional answer good enough to decide whether the idea is worth modelling properly. Its important property is that it describes itself as a heuristic, because a heuristic presenting itself as a simulation carries unearned authority into a document.

The simulation is the right tool for a decision that will be reviewed. It carries the scenario record, the branching, the lineage, and the comparison machinery described above. Heavy runs execute out of band, with asynchronous job handling and separate compute for the expensive families, so a substantial run does not have to be squeezed into the shape of a web request.

If the output is going into a document that a contracting officer, a board, or a public comment period will read, use the simulation.

Why does building on public federal data matter in a public process?

Because a reviewer can obtain the inputs and check you, and an argument you can lose on the evidence is the only kind worth making in a public process. FreightCortex is built entirely on public United States federal data.

Three sources carry most of it. The Freight Analysis Framework, currently FAF5, provides roughly 5.7 million flow records covering movements by origin, destination, mode, and commodity type. The Commodity Flow Survey public-use microdata supplies the shipment-level structure behind those flows. The National Highway Freight Network defines the highway designations corridor work is described against. No non-disclosure agreement, contract, or commercial licence is required for any of them.

The effect shows up during review. When a stakeholder disputes a base-year volume, the conversation is about a public table both parties can open, and an agency that wants to reproduce your work with its own analysts has nothing procedural in the way. Proprietary data can be better data, and where a result must survive public comment, a source everyone can reach beats one only you can reach. That tradeoff is worked through in the FAF5 decision dataset.

Where this goes wrong

Comparing runs that do not share their units. The symptom is a comparison table where one alternative looks implausibly strong. The cause is usually a mismatch in FAF zone geography, commodity aggregation, or horizon year. Put the shared fields into the comparison output, so a mismatch shows up as a mismatch rather than as a policy finding.

Branching without recording the change. The symptom is a variant nobody can describe six weeks later. The cause is copying a scenario and editing several things at once. One change per branch, written into the record as you make it.

Treating the heuristic as the simulation. The symptom is a single-lane elasticity number in a document that discusses network effects. The cause is that the heuristic answered immediately and the answer looked reasonable. Check which endpoint produced a number before it enters a deliverable.

Reporting a Monte Carlo mean. The symptom is a confident point estimate from a run whose whole purpose was to quantify uncertainty. Report the distribution, and say plainly what the unfavourable tail looks like.

Expecting nationwide multimodal network assignment. Full national network assignment is not a generally available capability in FreightCortex today. Scope corridor and lane questions to what the families above answer, and state that scope in the methodology rather than letting a reader infer more coverage than exists.

Common questions

What is the difference between a scenario and a model run?

A scenario is the stored record of assumptions, shocks, and horizon year. A run is one execution of that record through a simulation family. One scenario can be run repeatedly, for example after an input dataset is corrected, and the record is what keeps those runs comparable rather than merely sequential.

Do I need to know which simulation family to use before I start?

No, but you do need to know which question you are asking, because the family follows from it. Mode shift from a price change is cross elasticity or nested logit. Exposure to an uncertain input is Monte Carlo. The effect of a closure is network disruption. Writing the question in one sentence usually selects the family for you.

Can an AI agent run these simulations directly?

Yes. The seven families are callable through an MCP server as well as the API, so an agent can request a run with structured arguments. The scenario discipline still applies, and arguably applies harder: an agent that creates variants without lineage builds an uncomparable pile faster than a person can.

Is public federal data good enough for a policy decision?

For most freight questions at corridor and lane level, yes, and it carries an advantage proprietary data does not: your reviewer can obtain it. FAF5, the Commodity Flow Survey public-use microdata, and the National Highway Freight Network are standard reference sources for this work. Where a question needs facility-level or shipper-level detail, say so rather than modelling past the data.

What does a sensitivity sweep tell me that a comparison does not?

A comparison tells you which alternative is better under the assumptions you set. A sweep tells you how far those assumptions can move before the ranking changes. Decision makers usually need both, and the sweep is often what settles the discussion, because it turns an argument about assumptions into a statement about the range across which the recommendation holds.

Next steps

Take a question you are working on now and write it as a baseline scenario plus the variants that answer it, with the horizon year and shocks stated, before running anything. If you cannot write each variant as one change, the question is not yet specific enough to model, which is worth discovering before the runs.

For the uncertainty side, Monte Carlo freight forecasts covers how to read a distribution and what a well-formed input distribution requires. For mode shift specifically, cross-elasticity and mode shift works through what an elasticity result does and does not license you to claim. If your question is about a chokepoint rather than a policy lever, freight corridor bottleneck analytics starts from the network side.

The scenario record, the branching, and the comparison primitives described here run in production behind FreightCortex, whose entire data foundation is public federal data so that a reviewer can reproduce a result without asking anyone for access. More engineering writing sits in the Signals index.