A realistic analysis is rarely a single linear chain. Several outputs may share upstream work, different datasets may follow different paths, and systematic variations may affect only part of the computation.
Flow makes these relationships explicit in the logical graph and execution plan before processing scientific data.
flowchart LR
Source["<b>Source product</b>"]:::source
Shared["<b>Shared computation</b>"]:::transform
Nominal["<b>Nominal path</b>"]:::transform
Variation["<b>Applicable variation</b>"]:::transform
Product["<b>Reduced product</b>"]:::artifact
Source --> Shared
Shared --> Nominal --> Product
Shared --> Variation --> Product
classDef input fill:#f7f7f7,stroke:#5f6368,stroke-width:2px,color:#111111;
classDef flow fill:#e8f0fe,stroke:#3c6fbd,stroke-width:2px,color:#111111;
classDef source fill:#e7f5ff,stroke:#1c7ed6,stroke-width:2px,color:#111111;
classDef transform fill:#fff9db,stroke:#f08c00,stroke-width:2px,color:#111111;
classDef sink fill:#ebfbee,stroke:#2f9e44,stroke-width:2px,color:#111111;
classDef artifact fill:#eadcf8,stroke:#7950a3,stroke-width:2px,color:#111111;
classDef runtime fill:#fce8d5,stroke:#b56b22,stroke-width:2px,color:#111111;
classDef capability fill:#f7f7f7,stroke:#5f6368,stroke-width:2px,color:#111111;
Dependencies determine the graph rather than the visual order of a YAML file. Operations can branch from a shared product, bind explicitly to a particular upstream product, or declare graph dependencies when ordering is needed without moving scientific data.
Compilation records those relationships so that shared work does not need to be described repeatedly and inactive paths can be excluded deliberately.
For the complete graph model, see the Flow graph-structure documentation.
Systematic variations describe controlled alternatives to the nominal workflow. They may alter weights, fields, or datasets. Applicability and dependency information determine which graph paths need to be expanded; an unrelated branch does not have to be duplicated merely because another branch varies.
The Workshop contains runnable examples of independent variations and inline variation matrices in its systematics section.
Operations communicate through named products. A product can exist at different execution scopes:
Product handlers provide type-specific merge and materialisation behaviour. This allows Flow to orchestrate products such as histograms, cutflows, tables, or metadata without embedding their scientific implementation in the runtime.
See the Flow runtime documentation for the detailed lifecycle.
Products are values passed between operations. Artifacts are durable outputs of compilation or execution: for example a plan, graph source, histogram, skim, plot, report, manifest, or provenance record.
Writers and renderers claim their output paths during compilation so collisions can be found before execution. Manifests connect produced files to datasets, partitions, attempts, and provenance records. The output directory therefore contains both scientific results and evidence describing how they were produced.
For the artifact boundary and lifecycle, see the Flow artifact documentation.