FAST-HEP is a toolkit for building and executing sustainable High Energy Physics (HEP) analysis workflows.

Users describe the scientific workflow they want to perform without coupling it to a particular implementation or execution environment. Flow assembles and validates this description, constructs a logical graph, and compiles it into an execution plan. Registered capabilities then provide the data processing, inspection, provenance, and visualisation capabilities used throughout compilation and execution.

flowchart LR
    Workflow["<b>workflow.yaml</b><br/>scientific workflow"]:::input
    Flow["<b>Flow</b><br/>compile + plan"]:::flow
    Runtime["<b>Execution</b><br/>registered capabilities"]:::runtime
    Artifacts["<b>Artifacts</b><br/>results + provenance"]:::artifact

    Workflow --> Flow --> Runtime --> Artifacts

classDef input fill:#f7f7f7,stroke:#5f6368,stroke-width:2px,color:#111111;
classDef flow fill:#e8f0fe,stroke:#3c6fbd,stroke-width:2px,color:#111111;
classDef source fill:#e7f5ff,stroke:#1c7ed6,stroke-width:2px,color:#111111;
classDef transform fill:#fff9db,stroke:#f08c00,stroke-width:2px,color:#111111;
classDef sink fill:#ebfbee,stroke:#2f9e44,stroke-width:2px,color:#111111;
classDef artifact fill:#eadcf8,stroke:#7950a3,stroke-width:2px,color:#111111;
classDef runtime fill:#fce8d5,stroke:#b56b22,stroke-width:2px,color:#111111;
classDef capability fill:#f7f7f7,stroke:#5f6368,stroke-width:2px,color:#111111;

The resulting intermediate representations make the workflow inspectable before it is executed. Dependencies can be checked early, implementations can be replaced without rewriting the scientific workflow, and execution can be mapped onto different computing environments.

Toolkit overview#

FAST-HEP is built from focused packages connected through explicit contracts. Flow provides the general workflow, compilation, planning, and orchestration machinery. Other packages register capabilities that can participate in compilation and execution without becoming part of Flow itself.

flowchart LR
    Workflow["<b>workflow.yaml</b><br/>workflow description"]:::input

    subgraph FlowSystem["Flow"]
        direction LR
        Compile["<b>Compilation</b><br/>normalise, analyse,<br/>validate"]:::flow
        Plan["<b>Execution plan</b><br/>resolved operations + dependencies"]:::plan
        Runtime["<b>Execution</b><br/>orchestrated runtime"]:::runtime

        Compile --> Plan --> Runtime
    end

    subgraph Capabilities["Registered capabilities"]
        direction TB
        Carpenter["<b>Carpenter</b><br/>read, transform,<br/>aggregate, write"]:::capability
        Curator["<b>Curator</b><br/>inspect, validate,<br/>record evidence"]:::capability
        Render["<b>Render</b><br/>plots, reports,<br/>visualisation"]:::capability
        Extensions["<b>Extensions</b><br/>custom capabilities"]:::capability
    end

    Artifacts["<b>Artifacts + provenance</b>"]:::artifact

    Workflow --> Compile

    Capabilities -. "specifications" .-> Compile
    Runtime -. "implementations" .-> Capabilities

    Runtime --> Artifacts

classDef input fill:#f7f7f7,stroke:#5f6368,stroke-width:2px,color:#111111;
classDef flow fill:#e8f0fe,stroke:#3c6fbd,stroke-width:2px,color:#111111;
classDef source fill:#e7f5ff,stroke:#1c7ed6,stroke-width:2px,color:#111111;
classDef transform fill:#fff9db,stroke:#f08c00,stroke-width:2px,color:#111111;
classDef sink fill:#ebfbee,stroke:#2f9e44,stroke-width:2px,color:#111111;
classDef artifact fill:#eadcf8,stroke:#7950a3,stroke-width:2px,color:#111111;
classDef runtime fill:#fce8d5,stroke:#b56b22,stroke-width:2px,color:#111111;
classDef capability fill:#f7f7f7,stroke:#5f6368,stroke-width:2px,color:#111111;

This separation is deliberate: data readers, transforms, writers, inspectors, renderers, and execution technologies should be replaceable without requiring changes throughout the framework.

Packages#

The FAST-HEP toolkit currently includes:

Installation#

FAST-HEP packages are published independently and can also be installed through the fasthep meta-package.

pip install "fasthep[hep]"

For tutorials and reproducible example environments, the workshop material uses Pixi.

For development across the full toolkit, see fasthep-dev.

Distributed execution is provided separately by fasthep-distributed; it is not currently included in the fasthep[hep] installation bundle.

Tutorials and examples#

The fasthep-workshop provides runnable tutorials and example analyses.

The early tutorials build a small complete analysis from reading data through transformations, histograms, cutflows, and rendering. Later tutorials extend that foundation with saved datasets, provenance, systematic variations, workflow organisation, and selected distributed-execution paths. Further accelerator, extension, and complete case-study material is still being added.

For a first introduction to FAST-HEP, this is the best place to start.

Documentation#

Documentation is split according to purpose:

• This site — overview of the FAST-HEP toolkit, architecture, and project-level guides
• fasthep-flow — workflow description, compilation, planning, orchestration, and execution
• fasthep-workshop — tutorials, examples, and training material
• Package documentation — API references and package-specific details

Contributing#

FAST-HEP is developed openly on GitHub, and contributions are welcome across code, documentation, examples, testing, and infrastructure.

See the contributing guide for the development workflow and toolkit structure.

Get started#

New to FAST-HEP?

Start with the Getting Started guide for a first walkthrough of the toolkit, from installation to running a small analysis workflow.