Reading the results

Every example on this site ends by running a model and reading what came out. This page explains that output field by field, so those readings are checkable rather than taken on trust.

Any term here that you have not met before is in the glossary.

Two commands, two documents

command	what it gives you	when to use it
cwright simulate	one run, every detail, no error bars	understanding a model, the examples on this site
cwright run	many runs, each KPI as a confidence interval	making a decision

The difference matters. One run tells you what happened once. It cannot tell you whether a change you made was real or was the dice — for that you need run, which repeats the model with different random draws and reports a range.

tip

Use simulate while you are building the model and want to see what it does. Switch to run the moment you start comparing two versions of it.

The document simulate prints

Run the petrol station and this is what comes back. It is trimmed here only to keep the page readable — nothing has been renamed.

cwright simulate petrol-station.yaml --replication 0

{
  "model_hash": "6332672fdd928d24",
  "replication": 0,
  "released": 6651,
  "completed": 6640,
  "cycle_seconds_mean": 609.79,
  "sim_end_seconds": 691200.0,
  "events": 53228,
  "peak_live_entities": 37,
  "mean_in_transit": 0.0,
  "reworks": 0,
  "trace_hash": "cb0d44a2060a7519",
  "products": {
    "car": { "cycle_seconds_mean": 609.79, "n": 6640 }
  },
  "resources": {
    "pumps": {
      "utilisation_pct": 65.34,
      "availability_pct": 100.0,
      "mean_queue": 0.47,
      "mean_wait_seconds": 42.93,
      "served": 6651,
      "rejected": 0,
      "state_pct": {
        "Processing": 65.34, "Idle": 34.66, "Setup": 0.0, "Blocked": 0.0,
        "Load": 0.0, "Unload": 0.0, "ScheduledDown": 0.0,
        "UnscheduledDown": 0.0, "WaitingForLabour": 0.0
      }
    },
    "till": {
      "utilisation_pct": 83.3,
      "availability_pct": 100.0,
      "mean_queue": 2.8,
      "mean_wait_seconds": 253.62,
      "served": 6640,
      "rejected": 0,
      "state_pct": {
        "Processing": 83.3, "Idle": 16.7, "Setup": 0.0, "Blocked": 0.0,
        "Load": 0.0, "Unload": 0.0, "ScheduledDown": 0.0,
        "UnscheduledDown": 0.0, "WaitingForLabour": 0.0
      }
    }
  },
  "consumables": {
    "tank": {
      "mean_level": 14003.79,
      "min_level": 570.06,
      "final_level": 13572.39,
      "stockouts": 0,
      "mean_wait_seconds": 0.0,
      "total_taken": 298279.25,
      "total_replenished": 300000.0,
      "total_produced": 0.0,
      "total_spilled": 0.0,
      "waiting_at_end": 0
    }
  }
}

The whole-model numbers

field	what it means
released	How many entities the model let in.
completed	How many got all the way out.
rejected	⚠️ Entities turned away at a full queue, across the whole model. Zero unless some resource sets queue_capacity with when_full: "reject". Not the same field as the per-resource rejected below, which splits this by resource.
pieces_released / pieces_completed / pieces_rejected	The same three counts in pieces rather than entities. ⭐ These are what make the piece conservation identity checkable from a results document alone — see below.
cycle_seconds_mean	Mean cycle time — arrival to departure, waiting included.
sim_end_seconds	When the run stopped. This is warmup_seconds + horizon_seconds.
events	How many things the engine had to process. A rough measure of how much work the run was.
peak_live_entities	The most entities in the system at once — the high-water mark of WIP. ⚠️ The engine holds at most 1,048,575 at once and refuses a run that needs more, rather than confusing one entity with another; the usual cause is a store nothing feeds, which lint reports as unfed-store.
mean_in_transit	Mean number of entities in transport between steps, holding no machine. ⭐ Like every other mean here, it is taken over the measured window — warmup_seconds is discarded from it. ⚠️ It is a population, not a duration: at λ releases a second and a mean travel of T seconds it is λT, so it does not move when you rescale the clock.
reworks	How many entities were sent back to repeat a step.
abandoned	How many entities gave up waiting and left without being served. ⚠️ A lost sale. See patience_seconds on a step.
pieces_abandoned	The same, counted in pieces rather than entities.
lots_scrapped	How many entities were destroyed whole.
pieces_scrapped	How many individual pieces were destroyed, including those thinned from entities that survived.
preemptions	How many times work in progress lost its machine to something more urgent. Zero unless your model asks for preemption, which is how you check it is not happening by accident.
stalled_resources	⚠️ Resources that spent most of the run unable to make progress — a campaign waiting for work only it can produce, a blocked cycle, or a batch pool that never fills. Empty on a healthy run — see below.
break_concurrency	Per operator pool, the share of time exactly k people were on a break, for k = 0, 1, 2, ….
pieces_in_by_step	Per product, how many pieces finished each step of its route. ⚠️ Indices at or past the declared step count are rework excursion steps, appended in declaration order.
pieces_scrapped_by_step	Per product, how many pieces were destroyed at each step.
timers_dropped	⚠️ Queue timers still running when the clock stopped, so their window never closed. They are not counted as violations — nobody knows whether they would have breached — and a large number here means the horizon is too short to judge the timers you set.
model_hash	Identifies the model document exactly. Change one number in the model and this changes.
trace_hash	Identifies the run. Two runs with the same trace hash executed the same events in the same order.

⚠️ stalled_resources is an alarm, and an empty list is the good news

A resource can be stuck for ever while every other number in the document looks reasonable. There are three ways it happens, and they look identical from outside.

A campaign minimum on a re-entrant route. The only work that would release the group is work that same group has to run first, so it waits for something it is itself preventing.

A blocked cycle. With when_full: block each group holds a finished entity until the next has room — so a cycle of groups can each hold one and wait for the next, for ever. ⚠️ In a re-entrant plant that cycle is the ordinary topology, not an exotic one: two tools with opposite routes is the smallest there is. Measured on exactly that, both buffers of one, over 200,000 s: 200,212 released and 26 completed, both groups above 99.9% of their time in Blocked.

A batch pool that never fills. A minimum the arrivals cannot reach is a deadlock with no error, and it is the quietest of the three: the machine's servers are Idle, not blocked and not refusing work, because the waiting lots are not in the queue at all — they are in the batch pool, waiting for company that is not coming. Measured on a furnace needing 1,000 pieces against arrivals that deliver 200: 200 lots released, zero completed, and every state percentage entirely plausible for a quiet tool.

⭐ This one is reported whether or not the step declares a timeout_seconds, so a mis-specified batch group is loud even in a model that never asked for a deadline. The threshold is a share of the measured window rather than an absolute wait, which is what makes it need no guess about a "plausible" arrival rate: a healthy pool's wait is set by the arrival rate and stays put as the run lengthens — measured on SMT2020's HVLM fab, 5,817 s, 6,189 s and 6,670 s at run lengths of 5, 30 and 365 days, never above 1.35% of the window — while a stalled pool's wait grows with the run.

Nothing else says so. The model is valid, entities are conserved, the state percentages add to 100%, and the group genuinely is blocked rather than broken. stalled_resources names it directly.

⭐ A zero and an empty list are reported rather than hidden. If a key only appeared when something went wrong, you could never tell "nothing went wrong" from "this engine does not look". The same applies to abandoned and preemptions: seeing 0 is the result.

⭐ Why the per-step counts are separate numbers

A route with three steps each scrapping 5% of pieces destroys 1 - 0.95³ = 14.26% of a lot. A route with one step scrapping 14.26% destroys exactly the same. Every other number is identical — the same total scrapped, the same throughput, the same cycle time — so a scrap probability written on the wrong step, or written once where the route means three times, does not show up anywhere else.

pieces_in_by_step and pieces_scrapped_by_step are what separate them, and dividing one by the other gives each step's rate, which you can check against the probability your own model declares.

⚠️ Counted per visit. A re-entrant route that passes a step twice counts it twice, which is the right denominator for a per-visit probability.

⭐ Why break_concurrency is a distribution, not an average

A pool of four averaging one person away could be one person always away, or the whole pool away together for a quarter of the time. Those are very different shops and they have identical availability.

Staggering (first_at_seconds on an operator break) exists to prevent the second. This is how you check it worked: if breaks are properly spread, the share of time with two or more people away is zero.

The first thing to check, every time

released against completed. Here it is 6,651 against 6,640 — eleven cars short, which is simply the cars still at the pumps when the clock stopped. That is normal and expected.

A large gap is not. If a thousand entities went in and six hundred came out, something is stuck, and every timing number below is describing a system that is not keeping up. Check this before believing anything else.

warning

This check has caught real defects in this engine more than once. A model that quietly produces less does not crash and does not look wrong — utilisation, queue lengths and cycle time all stay perfectly plausible. Only conservation gives it away.

And the same identity in pieces. An entity carries pieces, and a step may scrap or abandon some of them without destroying the entity, so the entity counts above cannot tell you whether material went missing. pieces_released, pieces_completed and pieces_rejected sit beside pieces_scrapped and pieces_abandoned so the sum can be taken from the document itself:

pieces_released == pieces_completed + pieces_scrapped + pieces_abandoned + pieces_rejected
                   + whatever is still in flight

⭐ On a batching model in this repository's corpus that reads 51,375 released against 50,100 accounted for — a difference of 1,275, which is exactly the 51 lots of 25 pieces still inside the fab when the clock stopped. ⚠️ A remainder that is not a whole number of entities' worth of pieces is a leak, and no other field in the document would show it.

Per resource

Everything in resources is keyed by the id you gave it in the model.

field	what it means
utilisation_pct	Share of available time spent working. See the warning in the glossary — this is busy-over-available, not busy-over-total.
availability_pct	Share of total time the resource was able to work at all: not broken, not in maintenance. 100% here because the petrol station declares no downtime.
mean_queue	How many entities were waiting, on average.
mean_population_by_product	Per product, how many of its entities were here on average — queued or in service — which is what Little's law divides by throughput to get the time spent at this resource. A batch's members all count while it holds the tool; a lot still waiting for its batch to form does not.
mean_wait_seconds	How long each one waited, on average, over every arrival — including the ones that never waited. The textbook E[Wq].
servers	How many machines the group holds. ⭐ Reported because state_pct is a mean across a group's servers: four servers at 50% Processing is four times the machine time of one server at 50%, and the percentage alone cannot say which you have.
served	How many entities this resource finished.
rejected	How many were turned away because the queue was full. Only non-zero if you set a queue limit.
state_pct	Where the time went, adding to 100%.
down_concurrency	Share of time exactly k of this resource's servers were down at once, for k = 0, 1, 2, …. ⚠️ A distribution, not an average, for the same reason as break_concurrency: two tools down half the time each is a very different plant from both tools down together half the time, and a mean cannot tell them apart.
campaigns	How many campaigns ran — a campaign being an unbroken sequence of lots on one setup. Zero unless the resource declares min_run.
mean_campaign / min_campaign	The mean and shortest campaign length, in lots. ⭐ This is what separates a campaign minimum from a setup-avoidance rule. Both push state_pct's setup share the same way; only the distribution of campaign lengths says which mechanism produced it.
campaign_blocked_pct	Share of time a server refused work it could have taken, because it still owed a campaign. ⚠️ The instrument for min_run's one hazard — see resources.
commissioned	Stations a step's commissions put into service, over the whole run — a machine built in the warm-up is still standing afterwards.
built	Stations that exist to be paid for: installed plus commissioned on a group that grows, every station otherwise. ⭐ What resources.<id>.servers prices.
commissions_unfilled	Commissions that found no uninstalled station left: a build plan asking for more machines than servers allows. ⚠️ Counted, never silent.
capacity_engagements	How many times a station was called in by a capacity_policy. ⭐ Printed for every resource, zero included — a reader who never sees the key cannot tell nothing happened from this engine does not say. ⚠️ Not a station count: a policy that flaps reports many engagements of few stations, which is what min_engaged_seconds exists to bound.
queue_emptyings	How many times this resource's waiting queue reached zero. ⚠️ The queue, not the resource: the waiting count drops when a lot is dispatched, so a machine with one lot in service and nothing behind it has an empty queue.
queue_first_empty_seconds	When it first happened, in seconds from the start of the measured window. ⭐ This is the first-passage time — the answer to "which of these queues ran dry first, and when".
queue_empty_gap_mean_seconds	Mean seconds between one emptying and the next.

0.0 is ambiguous, and it takes queue_emptyings and mean_queue to read it

Three cases produce queue_first_empty_seconds of 0.0, and the three other fields are what tell them apart. Measured on four models:

	queue_emptyings	mean_queue	what happened
emptied at the first instant of the window	≥ 1	—	genuinely 0.0
the queue was empty for the whole window	0	0.0	no transition to empty, so nothing to time
the queue never emptied	0	> 0	it was occupied throughout

So a count of zero does not distinguish "empty the whole time" from "never empty" — both are the absence of a transition — and mean_queue is what separates them. A mean gap of 0.0 likewise means fewer than two emptyings, not no time between them.

A same-tick dip does not count. When several entities are released at one instant the engine serialises them, so the first is dispatched before the second arrives and the queue reads zero for zero simulated time — which depends on the order the engine happens to pick and is given zero weight by mean_queue on the line above. An emptying is a zero the queue stays at.

On a batching step, the lots waiting for company are not in this queue

A step that declares batch holds its lots in a batch pool until a load forms; only a formed batch queues for the machine. So on a furnace these three fields describe the queue of formed loads, not the lots waiting to fill one — the same exclusion mean_queue has. Measured on a furnace that batches: 504 emptyings roughly every 20 minutes, mean_queue 0.0, while the lots in the pool waited 2.3 hours each on average.

The pool's own wait is batch_wait_seconds_total over batch_entries, at the top level. If you are asking "did this furnace run out of work", that is the pair to read.

The three fields above are the whole of what a results document carries. The instants themselves — every tick each queue reached zero — are one number per emptying, which is memory proportional to the run on a plant that empties a queue on nearly every dispatch, so they are off by default and ⚠️ they have no command-line route today: they are reachable only by embedding the engine and calling Compiled::recording_emptyings(true). Unlike the flow records behind cwright trace, nothing writes them to a file.

state_pct is the one that answers "why"

utilisation_pct tells you a resource is busy. state_pct tells you what it was busy with, and the two are different questions:

state	meaning
Processing	Doing the actual work.
Idle	Nothing to do.
Setup	Doing a changeover — occupied, but producing nothing.
Load / Unload	Being loaded or unloaded, when those are modelled separately.
Blocked	Finished, but unable to hand the entity on because the next place is full.
ScheduledDown	Planned maintenance.
UnscheduledDown	Broken.
WaitingForLabour	Free, with work waiting, but no operator available.

A resource at 90% with 85% in Processing is genuinely busy. A resource at 90% with 45% in Setup is thrashing, and the fix is scheduling rather than capital. The paint shop is exactly that case, and the difference between the two readings there is a factor of sixty.

Per consumable

field	what it means
mean_level	Average stock on hand, weighted by how long it was at each level.
min_level	The lowest it ever got.
final_level	Stock at the end of the run.
stockouts	How many times an entity needed stock and had to wait.
mean_wait_seconds	Mean wait for stock over every claim, including the ones served instantly — the same definition as a resource's, and the one the textbook E[Wq] uses. ⚠️⚠️ So stockouts × mean_wait_seconds is not the total time spent waiting, and is wrong by whatever share of claims never waited at all. For the conditional mean, divide the total by stockouts yourself.
level_variance	Time-weighted variance of the stock level. ⭐ The mean alone cannot distinguish a store that sat at its average from one that swung between full and empty, and it is the variance — not the mean — that decides whether a safety stock is enough.
total_taken / total_replenished	Consumed and delivered over the run.
total_produced	Stock a step put back, through produces — as opposed to stock a purchase order delivered. ⭐ Kept separate from total_replenished deliberately: "what we bought" and "what we made or moved" are different questions with different owners, and a shelf filled by a stacker from a storeroom has the second and none of the first. Summing them would make a store nobody has to reorder look like one that is reordered constantly.
total_spilled	Production that did not fit and was lost at the cap. ⚠️⚠️ A store never exceeds its capacity, so a step producing into a full one loses the excess. A non-zero value is a real statement about the model — whatever fills this store is outrunning whatever drains it, and those units went nowhere. ⭐ Reported rather than silently discarded, and deliberately not part of the conservation identity below: it is stock that never entered the store, so counting it there would make a spilling model read as a leaking one.
waiting_at_end	Entities still queued for stock when the clock stopped.

tip

The conservation identity, which the engine asserts on every model:

baseline + total_replenished + total_produced - total_taken == final_level

⭐ And across two tiers it sharpens into a check with no tolerance at all — every case the storeroom gives up either lands on the shelf or is lost at its cap:

total_taken(storeroom) == total_produced(shelf) + total_spilled(shelf)

⚠️ With one boundary term, and it is worth knowing about: consumes draws when a step starts and produces puts back when it completes, so a restock task still working when the run ends is holding stock it has taken and not yet delivered. The remainder is always a whole number of whatever a task moves.

warning

mean_level cannot tell you whether you nearly ran out, and neither can final_level. The petrol station's tank averages 14,004 litres out of 30,000, which sounds comfortable, and ends the run at 13,572, which sounds fine. It got down to 570 litres — under 2% of the tank. Only min_level says so.

Per operator pool

Keyed by the pool id, for every operator pool the model declares.

field	what it means
people	How many are in the pool. The number you would change to test a staffing decision.
utilisation_pct	Share of available time the pool spent held by work — the same busy-over-available definition a resource uses, so breaks do not count against it.
availability_pct	Share of total time the pool was on shift at all, rather than on a break.
mean_wait_seconds	⚠️⚠️ How long work waited for a person, averaged over the requests that had to wait at all — not over every request. A pool is the one place in this document where that is so: a resource's and a store's are over every arrival, and this one is conditional, because the engine counts a request into both sums only when it actually joins the queue. ⭐ For a step that names one pool, recover the textbook E[Wq] by multiplying by the share that waited — for an M/G/1 pool that share is the pool's own utilisation. Measured on a single-operator pool at 50% load: this field reads 19.963 s where the unconditional E[Wq] is 10.000 s, a factor of exactly 1/rho. ⚠️⚠️ For a step that accepts SEVERAL pools there is no such recipe, and the number is smaller than the wait it names: the engine queues a token on every eligible pool, so each one counts the request in its denominator, while only the pool that serves records the time. Measured on one step accepting two single-person pools, both read about 5.02 s and the real conditional wait is their sum, 10.05 s. Read peak_queue alongside it.
engaged_seconds	⭐ Person-seconds the pool actually spent held by work, summed over everyone in it. The quantity a labour charge is priced on.
peak_queue	⭐ The most work waiting for this pool at once.
waiting_at_end	Requests still queued for a person when the clock stopped.

Two of these means are over every arrival and one is not

resources.*.mean_wait_seconds and consumables.*.mean_wait_seconds are averaged over every arrival, zero waits included — the textbook E[Wq]. operators.*.mean_wait_seconds is averaged over the requests that queued.

⚠️ An earlier version of this page said all three agreed, and said so in the words "as everywhere else in this document" — a claim that reads as settled precisely because it asserts its own consistency. The engine's labour module documented the conditional form correctly throughout; only the page was wrong. It was found by a Pollaczek–Khinchine target missing by a factor of two.

engaged_seconds is reported because it cannot be derived

The obvious reconstruction is utilisation_pct x people x elapsed. ⚠️ It over-counts by exactly the break time, because utilisation_pct is busy over available time and elapsed is total time — the same two denominators that once hid a ten-point shortfall behind a one-point apparent match. A price needs the raw person-seconds, so the engine reports them rather than leaving a reader to multiply two percentages together.

peak_queue is the number that says whether labour is the binding constraint

A pool at 40% utilisation whose queue reached eleven is not a spare pool — it is a pool whose work arrives in bursts, and the eleven people who waited did not care that the average was comfortable. ⚠️ Utilisation is an average over the run and averages hide exactly the peaks a staffing decision is made about; peak_queue is what survives that averaging.

⚠️⚠️ n is a sample size, not a completion count

Cycle times are held in a bounded buffer: at most 65,536 observations, after which one in every two is dropped and the stride doubles. Memory must not be a function of the horizon, so a ten-year run costs what a ten-day run costs — and above the cap products.<p>.n stops being the number of entities that finished.

Measured on the development laptop (AMD Ryzen 7 7840U), a four-server line at one job a second for 400,000 s: 400,000 completed, products.p.n of 50,001, cycle_stride 8. ⭐ completed is exact and always has been; it is n that is a retained count, and n × cycle_stride comes back to cycle_seen — 400,008 against 400,000, the rounding of a stride that doubled mid-run.

simulate reports the stride and the raw count so the difference is visible rather than inferred:

field	what it means
cycle_stride	1 while every observation is kept. Above that, one in every cycle_stride survived.
cycle_seen	How many cycle times were observed before thinning.

Above the cap, a longer run buys no precision at all

⚠️ The usual advice for an undecided comparison — run it longer — stops working here, and not gradually. Once the stride is above 1 the retained count no longer grows, so tripling the horizon triples what is thrown away and the estimate does not sharpen. Measured on one class: 5.53% spread at 2,000,000 s, 3.09% at 8,000,000 s, and 7.66% at 24,000,000 s — three times the run for a worse answer.

⭐ Replications are the lever that still works, because each contributes an independent estimate. The arithmetic that tells you which case you are in is horizon × arrival rate against 65,536, and it needs no experiment.

Defining your own KPI

There is no separate place to declare KPIs. These are the names cwright run reports an interval for, and every one of them is a name cwright compare --kpi accepts:

KPI name	what it measures
cycle_seconds	Mean cycle time across everything.
cycle_seconds.<product>	Mean cycle time for one product.
completed, released	Throughput.
utilisation_pct.<resource>	One resource's utilisation.
availability_pct.<resource>	One resource's availability.
queue.<resource>	One resource's mean queue.
level.<consumable>	Mean stock.
stockouts.<consumable>	Stockout count.
stockout_wait_s.<consumable>	Mean wait for stock.
reworks	Rework passes.
lots_scrapped, pieces_scrapped	Scrap, by lot and by piece.
abandoned, pieces_abandoned	Entities that gave up waiting, by lot and by piece.
preemptions	Times a lot was pushed off a server.
queue_timer_violations	Breaches of a queue_timer ceiling.
timers_dropped	Queue timers abandoned unresolved. Reported only when non-zero.
money	⭐ The charges total, only when the model prices something. A net cost, so lower is better — which is compare's default.

⚠️ Not every number in the document above is a KPI. mean_wait_seconds, served, rejected, min_level, final_level, state_pct and the whole operators block appear in simulate output and are not addressable by name — including the till's queueing wait, which is this page's own headline. They are read from the document, not compared.

So the petrol station's queue at the till is queue.till, and its tank's low stock story is stockouts.tank. Running it four times:

cwright run petrol-station.yaml --replications 4

gives each of those as a point estimate with a lo and hi bound, rather than a bare number — the full shape is described in the output reference.

Steering by one of them

cwright compare takes a KPI name and two models, and answers whether the change actually helped:

cwright compare base.yaml candidate.json \
  --kpi cycle_seconds --indifference 120 --replications 20

--indifference is the part people skip and should not. It says how big a change is worth acting on — here, two minutes of a customer's visit. Without it, the tool would report a statistically real one-second improvement as a win, and one second is not a reason to buy a till.

The threshold must be in the KPI's own units, and on its own scale

--indifference is read in whatever the KPI is measured in. cycle_seconds is seconds, so 120 means two minutes. ⚠️ But queue.till is a count of people, whose entire scale here is 2.8 — so --kpi queue.till --indifference 30 declares every possible change meaningless, and does it confidently, with no error and no wide interval.

A threshold carried from another KPI, or from another model, is the single easiest way to get a decided and wrong answer. Re-derive it per KPI, or express it as a fraction of that KPI's own baseline.

The verdict comes back as one of four, and the third is the one that matters most:

verdict	meaning
Better / Worse	A real difference, bigger than your indifference threshold.
NoMeaningfulDifference	A real measurement, and the change is too small to care about.
InsufficientEvidence	Not decided. The run cannot tell these apart yet — run more replications.

warning

InsufficientEvidence is not a failure and it is not "no difference". Treating the two as the same thing is how a search stops at the wrong answer while reporting success. If you get it, the document tells you how many more replications would settle it — and expect to need more than it first says, since that estimate is made from the noise seen so far.